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I 

This  is  the  Final  Report  of  the  project.  When  the  work  was  started  it  1 

j 

was  anticipated  that  it  would  run  for  a  number  of  years ,  since  extensive  J 

A 

$ 

basic  research  would  be  required' to  attain  the  objectives.  Unfortunately ,  i 

I 

from  our  point  of  view,  the  decision  was  made  not  to  extend  the  funds  of  j 

the  project  beyond  those  appropriated  for  the  first  year.  As  a  consequence,  ‘ 

i 

it  has  not  been  possible  to  reach  the  objectives  of  the  project.  We  have,  j 

however,  made  some  progress,  and  this  report  is  essentially  a  presentation 
of  those  parts  of  our  completed  work  which  we  hope  may  have  some  permanent 
value . 

The  work  of  the  project  consisted  of  three  parts t 

1.  Development  of  a  general  theory  of  organized  jyste'ns  and  organized 
activities.  Considerable  progress  was  made  in  this  urea.  Appendix  I 
of  this  report  gives  a  formal  presentation  of  the  completed  portions 
of  this  part  of  the  work. 

2.  Studies  of  the  work  of  biologists  concerning  the  development  of 
organisms  with  the  hope  that  this  would  give  us  useful  insight  into 
programming  and  into  the  development  of  organized  activities.  Appendix 
II  is  a  presentation  of  what  we  have  to  contribute  along  these  lines  as 
a  consequence  of  the  project  work. 

3.  Work  with  computer  experts  of  the  Syracuse  University  Research 
Corporation  on  the  design  of  computers  with  the  purpose  of  applying  our 
organization  theory  to  computers,  particularly  computers  used  in  Navy 
planes.  It  was  our  plan  to  begin  by  applying  organization  theory  to 
microprogramming.  By  the  time  of  termination  of  the  project,  the 
Project  Director  had  learned  quite  a  bit  about  computer  design  and 
microprogramming,  but  we  did  not  yet  have  any  useful  output. 

We  regret  that  the  project  was  terminated  and  we  hope  that  it  will  be 
possible  to  continue  at  least  parts  of  the  work  in  the  future. 

Personnel  on  the  project  consisted  solely  of  the  Project  Director. 
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Elements,  States.  Constraints  and  Choice 


It  is  the  purpose  of  this  paper  to  try  to  discover  basic  concepts  and 
phenomena  of  organization,  and  to  see  how  a  knowledge  of  these  can  be  used 
to  get  a  better  understanding  of  nature. 

As  a  model  or  prototype  of  an  organized  system  we  shall  study  the  properties 
of  a  system  consisting  of  a  number,  say  n,  of  elements,  each  element  having  a 
number,  say  L,  of  different  possible  states  in  which  it  can  exist.  It  is  not 
necessary  that  the  n  elements  be  similar,  although  they  will  be  in  many  cases 
of  interest.  It  is  also  not  necessary  that  the  number  or  type  of  states  in 
which  the  elements  can  exist  should  be  Che  same.  Furthermore,  the  numbers  n 
and  L  need  not  be  finite,  nor  even  countable. 

Consider  a  system  which  has  n  distinguishable  elements.  They  may,  for 
example,  be  distinguishable  by  their  location,  or  their  shape,  or  their  size, 
or  their  color.  If  the  ith  element  of  the  system  has  different  states  in 
which  it  can  exist,  then  the  total  number  of  different  possible  states  of  the 
system  is  at  most 


n 


If  there  are  no  constraints  between  the  different  elements  of  the  system,  i.e., 
if  they  are  all  independent,  so  that  the  possibility  of  any  state  for  the  general 
ith  element  is  not  affected  by  the  states  in  which  the  other  elements  exist,  then 
in  Equation  (1-1)  gives  the  actual  number  of  different  possible  states  of  the 
system.  We  shall  call  the  unconstrained  choice  o_f  states  of  the  system. 


•»r«a 


The  statement  that  a  system  is  organized  means  that  there  are  constraints  on 
or  between  the  elements.  If  there  are  constraints  on  the  elements,  this  means 
that  one  or  more  of  the  have,  in  effect,  been  decreased.  If  there  are  con¬ 
straints  between  the  elements,  this  means  that  the  elements  are  not  Independent 
of  each  other.  This  dependence  will  rule  out  those  originally  possible  states 
of  the  system  which  are  inconsistent  with  the  constraints.  It  follows  that  the 
choice  of  possible  states  of  the  organized  system  is  less  than  tfu.  The  magnitude 
of  the  redaction  in  choice  is  one  of  the  most  significant  measures  of  the  organ¬ 
ization. 

Let  us  consider  an  example  of  how  the  constraints  of  a  physical  system  can 
reduce  choice.  Consider  an  electrostatic  system,  and  let  us  consider  the  choice 
of  possible  potential  distributions  in  the  system.  If  there  were  no  constraints 
on  the  system  of  which  we  were  aware,  we  would  have  to  start  with  the  possibility 
that  every  point  in  space  had  a  continuous  range  of  possible  potentials  extending 
from  -«  to  +«.  Let  us  first  restrict  the  problem  to  the  region  R  bounded  by  the 
surface  S  in  Fig.  1.  This  reduces  the  elements  of  the  system  to  the  points  in 


Let  us  next  require  that  the  potential  obey  Laplace's  equation  in  the  region  R 
and  on  the  surface  S.  This  is  an  enormous  reduction  in  choice  of  potential  dis¬ 
tributions,  because  it  rules  out  all  distributions  which  are  not  continuous 
functions  and  do  not  have  second  derivatives,  and  whose  second  derivatives  do 
not  obey  Laplace's  equation.  The  choice,  however,  is  still  infinite.  As  a 
final  constraint,  let  us  specify  the  distribution  of  the  potential  on  the 


I 


3 


boundary  S.  If  the  specified  boundary  conditions  are  consistent  with  L&place's 
equation,  then  the  choice  is  reduced  to  unity;  namely,  the  solution  of  the 
problem.  If  the  boundary  conditions  are  not  consistent  with  Laplace's  equation, 
the  choice  is  reduced  to  zero. 

i 

i 

A  constraint  of  special  interest  for  us  is  the  limitation  of  bandwidth  in 

i  i 

communication  systems.  Suppose  we  have  a  system  signal  waveshapes  as  a 
function  of  time.  If  there  are  no  constraints,  the 'uncountably  infinite  1 

number  of  points  on  the  axis  of  time  are  the  system  elements  of  the* signal 

!  I 

waveshapes.  If,  however,  the  bandwidth  is  limited  to  a  finite  range,  the 
sampling  theorem^  tells  us  that  the  number  of  system  elements  is  limited  to 


the  countably  Infinite  number  of  sample  points.  This : is  at  first  surprising 

i 

since  there  are  apparenr.ly  an  uncountably  infinite  number  of  frequencies  in  the 
finite  but  continuous  range  of  allowed  frequencies.  However,  in  the  specification 
of  frequency  range,  the  implicit  constraint  is  added  that  the  function  is  Fourier 

•  i 

transformable,  which  accounts  for  the  reduction  to  countability. 

The  sampling  theorem  of  information  theory  states  "If  a  function  contains 

no  frequencies  higher  than  W  cycles  per  second,  it  is  Completely  determined  by 

1  ! 

giving  its  ordinates  at  a  series  of  points  spaced  seconds  apart,  thp  series  i 

extending  throughout  the  time  domain."  A  section  of  duration  T  of  such  a  func¬ 


tion  will  then  contain 


n  -  2TW 


sample  points,  i.e.,  effectively  distinguishable  elements.  We  may  'alsd  say  that 
a  system  of  bandwidth  W  has  an  effective  limit  of  resolution  in  time  of  ^  . 

In  all  real  systems  there  are  limits  of  resolution  which  reduce  the  effec- 

I 

tlve  number  of  distinguishable  elements  and  the  effective  number  of  distinguish¬ 


able  states  per  element  to  a  discrete  and  finite  set  even  in  continuous  systems.  , 


*Ref.  [1]  p.  67.  , 

p 


I 


I 


The  actual  limit  of  resolution,  is  due  to  that  cause  which  is  responsible  for  the 
coarsest  resolution.  In  the  hypothetical  absence  of  that  cause,  some  other  cause 
would  still  limit  the  resolution,  but  at  a  finer  value.  Limited  resolution  of 
both  elements  and  their  states  is  caused  by  deterministic  constraints  and  by 
noise.  In  the  particular  case  of  a  high  gain  amplifier,  the  actual  cause  in 
limiting  resolution  of  distinguishable  elements  is  usually  bandwidth,  while  the 
actual  cause  in  limiting  resolution  of  distinguishable  states  per  element  is 
usually  noise. 

Stochastic  Constraints  and  Entropy 

The  constraints  which  we  have  discussed  so  far  may  be  called  deterministic 
constraints  They  include  the  forces  between  physical  objects  and,  in  fact,  most 
of  the  laws  of  classical  physics,  excepting  those  in  which  statistical  mechanics 
is  involved-  They  would  include  bandwidth  limitation,  as  already  mentioned 
Their  effect  on  choice  is  to  specify  chat  certain  choices  are  ruled  out,  i  e. , 
they  are  not  possible. 

An  equally  important  class  of  constraints  are  the  stochastic  constraints. 
Their  effect  is  not  to  rule  out  choices,  but  to  affect  their  probability  of 
occurrence.  In  a  formal  sense,  deterministic  constraints  are  a  special  case  of 
stochastic  constraints.  In  a  system  which  is  organized  completely  by  determin¬ 
istic  constraints,  there  is  unit  probability  for  a  particular  choice  (or  result) 
and  zero  probability  for  all  other  choices. 

Those  laws  of  quantum  physics  which  are  expressible  in  terms  of  the  wave 
function  are  stochastic  constraints.  Many  stochastic  constraints  are  expressions 
of  the  incomplete  knowledge  on  the  part  of  the  observer  concerning  the  initial 
conditions.  In  some  such  cases,  a  complete  knowledge  of  the  initial  conditions 
would  completely  determine  the  final  results.  On  the  other  hand,  the  presence 
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of  random  noise  in  a  system  represents  a  constraint  which  is  intrinsically 


stochastic. 


A  most  important  measure  of  stochastic  systems  is  entropy.  If  a  set  {p^} 
of  probabilities  satisfies  the  equation 

«> 


H  -  -£Pl 


J-og  P. 


is  defined  as  the  entropy  of  the  set  of  probabilities  . 

The  same  formal  definition  of  entropy  can  be  extended  to  apply  to  the 
states  of  an  element  or  the  states  of  a  system.  For  example,  suppose  that 
the  different  possible  states  of  the  itl1  element  are  equally  probable. 


1  fk 

p  ■  - —  «  probability  of  the  j  state 
J  Li 


Hi  -  p  log  p,  -  log  -  log(choice) 

jFT 


Entropy  equals  log(cholce)  only  when  all  the  different  possible  choices  are 
equally  probable.  Any  other  distribution  of  probabilities  will  give  a  smaller 
entropy.  Accordingly,  we  shall  consider  the  assignment  of  a  set  of  probabilities 
to  the  states  as  a  stochastic  constraint,  if  the  probabilities  are  not  all  equal. 

If  a  system  were  made  up  of  n  similar  elements,  between  which  there  were 
no  constraints,  then  there  would  be  a  choice  of 

WM  -  Ln  (7) 

different  possible  states  of  the  system.  If  these  states  were  all  equally 


xhe  numerical  value  of  H  will  depend  on  the  base  used  for  the  logarithms. 
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probable,  then  the  entropy  of  the  system  would  be 

H  *  log  «  n  log  L  »  log(choice)  (8) 

We  note  that  the  entropies  of  the  system  elements  are  additive  when  there  are 
no  constraints  between  them-  Every  constraint,  whether  acting  on  individual 
elements  or  between  elements,  will  reduce  entropy3 4.  The  change  in  entropy  is 
probably  the  most  important  measure  of  the  effect  of  a  constraint  . 

3 .  Organization  as  Negentropy 

from  the  point-of-view  of  entropy,  we  may  look  upon  the  organization  of  a 
system  as  a  reduction  in  its  entropy.  We  can  measure  the  organization  in  units 
of  entropy  and  we  say  that  the  amount  of  organization  is  equal  to  the  reduction 
in  entropy  caused  by  the  constraints  which  constitute  the  organization.  Accord¬ 
ingly,  we  say  that  organization  is  measured  in  units  of  negative  entropy,  which 
term,  following  Brillouin,  we  contract  to  negentropy  Thus,  uncertainty  is 
measured  in  units  of  entropy,  and  organization  is  measured  in  units  of  negentropy. 

4 .  Ambiguity  and  Noise 

If  a  system  is  organized  to  the  extent  that  each  system  element  is  in  the 
particular  state  prescribed  for  it,  then  the  probability  that  the  system  is  in 
its  prescribed  state  Is  unity  and  the  probability  that  it  is  in  any  other  state 
is  zero.  The  entropy  of  the  system  is  then  zero  and  no  further  organization  of 
this  system  is  possible. 

In  some  organized  systems,  it  may  not  be  desired  that  the  entropy  be  reduced 
to  zero.  For  example,  a  manufactured  clock  may  be  considered  completely  organized 
irrespective  of  the  time  indicated  on  its  face.  In  some  cases,  one  can  eliminate 
the  residual  entropy  by  redefining  all  these  equally  desirable  states  as  a  single 


3 Ref .  [2 j  p  17-19 

4 

The  same  measure  will  also  apply  to  the  effect  of  deterministic  constraints.  The 
states  v.'iiich  are  ruled  out  by  die  constraints  are  assigned  zero  probability.  All 
allowed  states  are  assigned  equal  probability  in  the  absence  of  stochastic  con¬ 
straints 


7 


state,  but  often  that  Is  not  practical.  In  many  cases,  however,  the  residual 
entropy  of  an  orga.^zed  system  represents,  what  is  in  principle  at  least,  a 
defect  in  organization.  This  is  true  in  the  important  case  when  there  is 
ambiguity  (noise)  present. 

We  shall  divide  the  conditions  in  which  ambiguity  is  present  into  two 
categories : 

(1)  When  one  or  more  of  the  elements  of  a  system  are  not  in  their  prescribed 
states,  we  say  that  state  noise  or  amplitude  noise  is  present”1.  We  also  use  these 
terms  to  describe  the  situation  in  whicn  there  is  ambiguity  as  to  which  states  the 
elements  are  in,  or  the  situation  in  which  the  characteristics  of  the  states  are 
not  those  prescribed, 

(2)  When  there  is  an  ambiguity  or  probability  of  misinterpretation  as  to 

which  element  of  a  system  is  which,  we  ribe  the  condition  by  saying  that 

£ 

identity  noise  or  location  noise  is  preset t  .  We  also  use  the  term  identity 
noise  to  describe  the  situation  in  which  an  element,  although  unambiguously  rec¬ 
ognized,  does  not  have  the  characteristics  which  are  prescribed  for  it. 

Example  (a) 

Radar  Signal  Organized  by  che  Target 

A9  an  example  of  amplitude  noise,  we  cite  the  case  of  the  signal  which  is 
displayed  on  a  radar  screen  (See  Fig  2)  the  signal  having  been  organized  by 
the  scattering  of  a  transmitted  electromagnetic  wave  by  one  or  more  targets. 

The  organized 

Fig-  2 

Radar  Signal 

Sample  values  indicated  by 


The  term  "amplitude  noise"  is  used  when  the  possible  states  of  each  element  form 
ordered  sets 

=The  term  "’ocation  noise"  is  used  wiien  the  elements  of  a  system  form  an  ordered 
set 


X. 
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signal  Is  used  Co  determine  Che  range  (distance  from  observer)  and  size  of  Che 
targets.  In  this  case  the  noise  arises  in  the  receiver  and  is  added  (as  a  com¬ 
plex  quantity  with  amplitude  and  phase)  to  the  signal  coming  from  the  target. 

The  elements  of  the  organized  signal  in  Fig.  2  are  the  sample  points  (the  signal 
has  limited  bandwidth)  and  their  possible  states  consist  of  the  continuous  range 
of  possible  amplitudes  at  the  sample  points.  It  is  known  from  the  characteristics 
of  the  system  that  the  noise  amplitude  at  each  sample  point  has  a  probability 


density 


N 


p (N)dN  -  —■  e^N/^ d 

<>> 


(9) 


where  N  is  the  noise  amplitude  and<^tl^>is  the  ensemble  average  of  its  square.  The 
phase  of  the  noise  is  uniformly  distributed  from  zero  to  2".  Figure  2  is  a  plot 
of  the  resultant  amplitude  when  the  complex  quantities  representing  signal  and 
noise  are  added.  The  figure  thus  illustrates  amplitude  noise*'.  There  appear  to 
be  targets  at  ranges  of  5  and  9  miles.  Later  on  we  shall  calculate  quantitatively 
the  effect  of  the  amplitude  noise  on  the  organization  of  the  signal. 

Example  (b) 

Doppler  Spread  of  a  Spectrum 

As  an  example  of  location  noise  we  cite  the  broadening  of  spectral  lines  due 
to  the  doppler  effect  caused  by  the  thermal  motion  of  the  radiating  atoms.  As  a 
consequence  of  doppler  broadening,  two  or  more  spectrum  lines  may  overlap  and  it 
may  not  be  possible  to  separate  (resolve)  them. 

Example  (c) 

The  Spread  Function  of  a  Photographic  Plate 

Another  example  of  location  noise  is  the  spread  function  of  a  photographic 
plate.  If  the  system  of  lenses  which  forms  an  image  in  a  photographic  plate  is 


The  use  of  the  term  "amplitude"  in  amplitude  noise  refers  to  a  state  magnitude 
and  the  magnitude  Is  not  necessarily  the  amplitude  of  a  complex  quantity,  al¬ 
though  it  happens  to  be  in  this  example. 
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so  good  that  it  may  be  considered  free  of  diffraction  and  aberration,  so  that 
a  point  on  the  object  is  transformed  into  an  optical  point  in  the  image,  the  image 
on  the  photographic  plate  of  a  point  in  the  object  will  still  have  a  finite  spread. 
The  reason  for  this  is  that  there  is  a  stochastic  component  in  the  photographic 

g 

process  which  spreads  the  image  point  on  the  plate  .  The  distribution  of  density 

on  the  plate  as  a  function  of  distance  from  the  light  point  in  the  image  is  called 

the  spread  function  of  the  plate.  It  is  the  resultant  of  the  effects  of  what  i3 

almost  an  ensemble  of  photons,  so  that  it  is  largely  determined  by  the  expectation 

9 

values  of  the  process  and  to  that,  extent  it  is  almost  deterministic.  Since,  the 
number  of  photons  involved,  however,  is  finite,  there  are  always  fluctuations 
present  in  any  particular  experimental  example  of  the  spread  function.  The  deter¬ 
ministic  or  average  values  of  the  spread  function  have  the  same  effect  on  image 
resolution  as  a  reduction  in  effective  spatial  bandwidth  such  as  would  be  caused 
by  diffraction  or  aberration. 

Actually,  the  observed  doppler  spread  of  a  spectrum  line  is  likewise 
principally  the  observation  of  the  expectation  of  the  doppler  effect  in  what  is 
essentially  an  ensemble  of  radiating  atoms.  The  noise  in  the  radar  signal  example, 
however,  is  much  more  like  the  effect  of  an  individual  stochastic  event,  since  the 
expectation  of  the  noise  can  be,  and  for  all  practical  purposes  is,  subtracted  out 
in  the  observation  process.  On  the  other  hand,  if  the  amplitude  noise  in  Fig.  2 
became  so  large  that  one  of  ths  noise  peaks  were  mistaken  for  a  target,  then  the 
amplitude  noise  would  give  rise  to  identity  noise.  This  would  be  an  example  of 
an  individual  stochastic  event  in  location  or  identity  noise  in  the  observation. 

Of  course,  there  would  also  be  errors  made  in  estimating  the  signal  amplitudes 
returned  by  the  actual  targets.  We  note  that  in  systems  in  which  both  the  dis¬ 
tinguishable  elements  and  their  states  are  in  ordered  sets,  a  small  amount  of 


8Ref .  (3]  p.  298. 

9 

This  is  an  oversimplification  since  the  real  process  is  non-linear. 
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location  noise  can  be  Interpreted  as  amplitude  noise  and  may  not  cause  ambiguity. 
This  is  illustrated  in  Figure  3. 


Fig  3 

AB  *  magnitude  of  location 
noise, 

DC  *  magnitude  of  equivalent 
amplitude  noise  at  the 
next  sample  point  (Note 
chat  it  is  proportional 
to  the  signal). 


There  is  one  type  of  noise  which  in  linear  and  quasi-linear  systems  represents 
Che  essence  of  disorganization.  This  is  white  gausslan  noise.  White  gaussian  noise 
is  a  type  of  amplitude  noise.  Identity  noise  and  location  noise,  when  the  effect 
of  the  latter  is  to  cause  ambiguity  between  elements  of  a  system,  are  of  an  essen¬ 
tially  non-linear  nature'1'0.  White  gaussian  noise  is  a  stochastic  disturbance  (of 
the  organization  of  a  system)  having  the  following  three  properties: 

(1)  It  has  a  gaussian  distribution  of  amplitudes,  i.e. 


P(N)dN  =  -1-  -  e  2°  a  (10) 

]/  2 no  2 


,+cc 


where  c 


N  p (N)dN 


(U) 


while  N  is  the  noise  amplitude  and  p(N)  is  its  probability  density. 

(2)  The  noise  amplitudes  of  the  different  elements  are  statistically 

independent . 

2 

(3)  o  is  the  same  for  every  element. 


Location  noise,  when  it  is  so  small  in  magnitude  that  it  causes  no  ambiguity 
between  system  elements,  is  not  identity  noise.  In  this  case  it  may  have  the 
effect  of  amplitude  (state)  noise  as  previously  noted. 
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For  a  given  amount  of  noise  energy  and  a  given  number  of  system  elements, 
white  gaussian  noise  has  the  maximum  possible  entropy. 

White  gaussian  noise  also  has  the  remarkable  property  that  it  has  a 

2 

statistically  independent  gaussian  distribution  of  the  same  a  in  any  ortho¬ 
normal  basis  system  of  vectors  in  function  space^.  This  mathematical  fact  is 
the  basis  of  the  equipartition  law  in  statistical  mechanics,  since  the  thermal 
motion  of  moelcules  has  the  characteristics  of  white  gaussian  noise. 

In  the  real  world,  white  gaussian  noise  arises  as  a  consequence  of  the 
central  limit  theorem  and  the  breakdown  of  constraints  between  system  elements 
in  the  superposition  of  large  numbers  of  independent  effectively  stochastic 
distrubances ,  In  the  particular  case  of  thermal  motion,  there  are  no  stochastic 
constraints  between  system  elements  because  of  the  random  nature  of  the  stochastic 
coupling  between  degrees  of  freedom,  and  the  gaussian  distribution  is  a  consequence 
of  the  central  limit  theorem. 

Our  original  definition  of  entropy  was 

H  “  ”/LPi  108  Pi  (12) 

i 


However,  when  a  system  has  a  continuous  distribution  of  states  as  is  the  case  for 
white  gaussian  noise,  this  definition  is  modified  to 


H 


p(x)  log  p (x) dx 


(13) 
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^Reference  (4]  p,  232. 

12 

Reference  [1]  Chap.  IV,  The  form  of  Eq .  (13)  neglects  what  amounts  to  an 
infinite  additive  constant  in  the  encropy  of  a  continuous  distribution.  It 
is  reasonable  to  do  this  only  because  in  all  practical  applications  of  the 
entropy  of  a  continuous  distribution,  such  as  in  Eq.  (17),  the  infinite 
constant  is  cancelled  by  the  subtraction  of  the  identical  constant. 
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where  Che  possible  staces  of  the  system  are  specified  as  values  of  x  and  Che 

13 

range  of  x  Is  from  a  Co  b.  It  is  easily  shown  that  the  entropy  of  white 
gaussian  noise  is  log  2ir  eo2  per  system  element.  The  noise  amplitudes  of  the 
different  system  elements  in  white  gaussian  noise  are  statistically  independent. 
Therefore,  if  the  system  has  n  elements,  the  total  entropy  of  the  white  gaussian 
noise  is 

H  “  n  log  2ir eo2  (14) 

6.  The  Effect  of  Amplitude  Noise  on  Choice  of  States 

We  can  use  Equation  (14)  to  obtain  an  intuitive  concept  of  the  effect  of 

noise  on  choice.  Suppose  we  have  a  communication  system  having  signals  of  duration 

T  and  bandwidth  Bj  whose  only  other  constraint  is  that  the  mean  square  amplitude 

2 

of  the  signals  (i.e- ,  the  average  power)  is  prescribed  as  S  .  The  instantaneous 
amplitude  may  be  either  positive  or  negative.  According  to  the  sampling  theorem, 
these  signals  will  have  2TB  sample  points  and  therefore  2TB  independent  elements. 
We  will  consider  the  case  in  which  2TB>>1.  Now  the  entropy  of  white  gaussian 
noise  is  the  maximum  of  that  of  all  ensembles  of  signals  of  the  same  mean  square 
amplitude.  Since  the  set  of  possible  signals  of  the  foregoing  communication 
system  has  no  constraints  other  than  bandwidth,  duration  and  average  power,  we 
shall  consider  Chat  its  entropy  is  the  same  as  though  it  were  a  random  noise 
ensemble^4.  Therefore,  prior  to  making  a  selection  of  one  particular  signal,  the 
entropy  of  the  set  is 

Hg  =  2TB  log  V2"eS2  (15) 


^Reference  [1]  pages  131-133 
14 

This  is  based  on  the  assumption  that  for  signals  of  the  same  average  power,  as 
2TB  becomes  very  large,  those  signals  whose  wave  shapes  approximate  white  gaus¬ 
sian  noise  will  ultimately  greatly  outnumber  all  others  combined.  This  appears 
to  be  a  reasonable  extrapolation  of  results  which  have  been  derived  in  informa¬ 
tion  theory-  See  Ref,  [2]  pages  15-16.  *• 
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Suppose  there  is  also  white  gaussian  noise  in  the  system,  with  average  power  a  . 

Now  the  superposition  of  two  gaussian  ensembles  gives  another  gaussian  ensemble 
with  a  mean  square  value  equal  to  the  sum  of  those  of  the  superimposed  ensembles. 
Therefore,  prior  to  the  selection  of  a  signal,  the  entropy  of  the  signal  plus 
noise  ensemble  is 

H$+n  -  2TB  log  ^e(S2+o2)  (15A) 

After  making  the  selection  of  a  particular  signal,  the  entropy  of  the  system 
is  reduced  to  that  of  the  noise,  namely 

■  2TB  log  ^2 neo2  (16) 

The  negentropy  of  the  organized  signal  in  the  presence  of  the  noise  is  therefore 

HS-fN  ~  HN  =  2TB  log  ‘  2TB  lo®V 2?Iea^  B  2TB  log  ^1  +  | 

Comparing  Eq.  (I7)  with  Eq.  (8)  we  see  that  the  effect  of  the  noise  is  to  reduce 

S  S 

the  number  of  states  per  element  to  the  finite  effective  number—,  if  — >>1-  In  other 

0  C 

words,  o  is  the  effective  spacing  between  distinguishable  states.  In  the  absence 
of  noise,  the  theoretical  number  of  states  per  element  is  uncountably  infinite. 

The  latter  fact  is  related  to  the  infinite  constant  'uentioned  in  the  footnote  to 
Eq.  (13), 

In  all  organized  systems  of  the  real  world  in  which  the  states  of  an  element 
have  a  continuous  range  of  possible  values,  there  is  effectively  always  noise 
present15  so  that  the  negentropy  measuring  the  organization  is  finite. 

Coherence  and  the  Effects  of  Location  Noise 

The  word  coherence  is  very  significant  in  physics  and  in  communication  theory. 
Things  which  are  coherent  are  parts  of,  or  are  expressions  of,  the  same  individu¬ 
ality  Their  characteristics  arise  from  the  same  source  Knowledge  of  one  is 


15 


Of  course,  the  noise  need  not  be  white  gaussian. 
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sufficient  to  determine  the  other.  Things  which  are  partially  coherent  are  par- 
1 6 

tially  dependent  In  accordance  with  this  terminology,  we  shall  use  the  degree 
of  coherence  to  describe  the  degree  of  dependence  between  the  parts  of  an  organized 
system.  If  a  system  is  completely  organized,  its  parts  are  completely  coherent. 

A  good  mathematical  example  of  an  entity  whose  parts  are  coherent  is  a  function 
which  can  be  expanded  in  a  Taylor  series  about  the  point  P  (see  Fig.  4),  the  ex¬ 
pansion  holding  everywhere  in  the  region  R,  If  the  values  of  this 

R 

Fig.  4 

function  are  known  in  any  small  region,  R' ,  no  matter  how  small,  the  function  is 
completely  determined  everywhere  in  R.  The  function  is  therefore  coherent  in  the 
region  R.  The  derivatives  of  all  order  in  the  Taylor  series  may  be  considered 
to  represent  the  set  of  system  elements  of  the  function.  They  are  a  countably 
infinite  set1^. 

Both  amplitude  and  location  noise  will  tend  to  reduce  coherence.  We  shall 
find  it  very  instructive  at  this  time  to  consider  some  effects  of  location  noise. 

In  Example  (c)  of  Section  4  we  pointed  out  that  location  noi3e  causes  a  unit  im¬ 
pulse  to  spread  to  a  function  of  finite  width;  which,  following  photographic 
terminology,  we  shall  call  the  spread  function.  The  effect  of  the  spread  function 
is  to  cause  a  loss  of  resolution  very  similar  to  that  caused  by  a  bandwidth  limi¬ 
tation.  This  limits  the  smallness  in  size  of  the  parts  of  a  system  which  can  be 
organized. 


In  optics,  coherence  refers  to  the  dependence  between  the  instantaneous  respective 
amplitudes  and  phases  of  two  disturbances  or  of  parts  of  the  same  disturbance. 

The  set  of  orthogonal  functions  in  an  orthogonal  function  expansion  of  the  ori¬ 
ginal  function  in  the  region  R  would  represent  an  alternative  set  of  system 
elements  of  the  same  system.  The  ;ransformatfon  of  the  representation  of  a  sys¬ 
tem  from  one  set  of  system  elements  to  another  is  often  very  useful. 


15 


Let  us  next  consider  the  same  system  represented  in  its  Fourier  transform 
domain.  The  elements  of  the  system  are  now  its  frequency  components  and  we  turn 
our  attention  to  the  effect  of  location  noise  of  these  frequency  components.  The 
doppler  spread  of  a  spectrum,  such  as  mentioned  in  Example  (b)  of  Section  4,  would 
represent  location  noise  in  the  frequency  domain.  Consider  a  single  frequency 
component  which  is  spread  by  location  noise  in  the  frequency  domain  to  cover  a 
frequency  range  Af .  In  the  absence  of  location  noise,  if  the  amplitude  and  phase 
of  the  frequency  component  were  known  at  one  point  in  time,  it  would  be  completely 
known  for  all  time.  However,  as  a  consequence  of  Che  location  noise,  if  the  am¬ 
plitude  and  phase  of  the  component  are  known  at  one  point,  they  gradually  become 
uncertain  as  one  leaves  that  point  and  become  completely  uncertain  for  distances 

from  the  point  exceeding  .  If  the  value  of  Af  is  the  same  for  all  frequency 
18 

components  ,  we  can  say  that  location  noise  limits  the  coherence  rar.ge  of  the 

signal  (or  other  system  which  can  be  represented  as  a  continuous  function)  to  a 

maximum  value  of  77 
At 

Consider  a  one  dimensional  system  which  has  location  noise  in  the  time  domain 
causing  a  spread  function  of  effective  width  At,  and  which  has  location  noise  in 
Che  frequency  domain  causing  a  spread  function  of  effective  width  Af.  Then  the 


This  would  usually  be  true  for  narrow  band  systems  such  as  a  modulated  carrier 
or  a  wave  packet  in  quantum  mechanics. 

j.9 

The  effect  of  the  resistive  component  of  a  tuned  circuit  is  analogous  to  the 
introduction  of  location  noise  at  the  resonant  frequency,  fQ,  and  its  amount 
Af  is  the  same  as  the  bandwidth  causad  by  the  resistance.  The  effective 

coherence  range  is  vr  ,  and  this  is 
A  t 


f 


£ 

,f 


Q 


cycles  at  the  tuning  frequency. 
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system  cannot  be  organized  in  parts  smaller  than  At  nor  larger  than  .  The 
number  of  effective  elements  of  the  organized  system  can  therefore  not  exceed 


(18) 


Af  •  At 

For  an  n-dlmensional  system,  it  is  fairly  obvious  that  there  would  be  a  factor 
of  the  same  form  as  Eq.  (18)  for  each  dimension.  This  is  a  significant  result. 

If  the  above  one  dimensional  system  is  limited  to  a  bandwidth  B  which  Is 
less  than  then  bandwidth  limitation  rather  than  location  noise  limits  the 
smallness  of  organization  and  the  number  of  effective  elements  of  the  organized 
system  cannot  exceed 


B_ 

Af 


(19) 


8 .  Requirements  for  Organizing  a  System;  Information,  Energy  and  Transducers 

We  next  consider  the  requirements  for  organizaing  a  system.  The  system  may 
be  either  a  static  structure  or  an  operating  mechanism.  For  example,  we  would 
usually  consider  a  building  as  a  static  structure.  Likewise,  an  automobile  on 
display  in  a  showroom  Is  a  static  structure.  On  the  other  hand,  a  running  auto¬ 
mobile  or  a  living  organism  is  an  operating  mechanism.  In  order  to  make  an 
organized  system,  one  must  somehow  obtain  its  elements  and  put  them  in  their 
proper  states.  Let  us  see  what  is  required  to  do  this.  For  example,  in  order  to 
make  an  automobile,  one  must 

(a)  Have  a  specification  of  the  automobile  to  be  made. 

(b)  Know  where  the  parts  or  raw  materials  are  available  fer  purchase. 

(c)  Have  the  necessary  facilities  to  process  them  into  the  organized 
automobile . 

(d)  Have  the  necessary  money  to  pay  for  the  purchase  and  manufacturing. 
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This  description  cf  the  organization  of  an  automobile  describes  the  require¬ 
ments  for  so  doing  in  the  socio-economic  world.  The  description  is  incomplete, 
but  is  sufficient  to  illustrate  certain  general  principles.  In  order  to  make  an 
automobile  in  the  socio-economic  world,  one  needs  information,  available  raw 
materials,  a  manufacturing  facility  and  money  to  pay  for  everything.  The  analo¬ 
gous  requirements  to  make  an  organlzad  system  in  the  physical  world  are  information, 
available  raw  material,  transducers  and  available  energy^. 

A  very  Instructive  example  of  what  is  involved  in  organizing  a  system  is 
provided  by  a  "matched"  filter  as  used  in  radar.  In  this  case  a  pulse  of  elec¬ 
tromagnetic  energy  of  some  specified  wave  shape  s(t)  is  radiated  from  a  trans¬ 
mitter.  If  some  of  the  energy  in  this  pulse  is  scattered  or  reflected  by  a 
target  and  returned  to  a  receiver  at  the  same  location  as  the  transmitter,  then 
the  detection  of  the  returned  energy  is  evidence  of  the  existence  of  the  target. 
Measurement  of  the  time  delay,  call  it  tQ,  between  pulse  transmission  and  reception, 
becomes  a  measurement  of  the  distance  between  the  target  and  the  radar,  since  the 
velocity  of  propagation  of  electromagnetic  energy  is  known.  Figure  5  shows  a  di¬ 
agram  of  the  waveshape  of  the  transmitted  radar  pulse,  as  well  as  that  of  the 
received  pulse  which  enters  the  radar  receiver.  The  radar  receiver  behaves  as  a 
matched  filter.  A  great  many  details  of  the  radar  equipment  which  are  not  pertin- 
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Y(u)  =  |Y (u>)  |e 


-■14  ((it)  ' 

|s(u>)  |e  S,-S*(u) 


which  Is  the  .complex  conjugate  of'S(io).  The  received  signal,  a^  (t-t^),  will 


have  a  foiirier  transform 


a,  j  s(t-tQ)e  J  cdt  ■  aS  (u)e 


The  output  of  the  matched  filter  willth4n  be 


Y(w)  •  S(u)e  .^e^utrdw 


€ 


j  jw(t-t  ) 

|s(u)  |  e  du> 


Since  the  fourier  transform  of  the  power  spectrum  is  the  autocorrelation  function  , 

,  J 

Eq.  (24)  ’tells  us  that  the  output  Bignal  has  the  wave ‘shape  of  the  autocorrelation 

function  of  s(t),  and  that  it  is  delayed  by  a  time  t^.  Furthermore,  if  white 

•  |  »  1  : 

gausslan  noise  is  introduced  at  the  input,  as  usually  occurs  in  a  high  gain  amp- 

i  r 

llfier,  and  if  the  White  gausslan  noise  has  a  noise  power  per  unit  bandwidth  of 
2  1  22 

4ir  k  ,  then  it  is  easily  shown  that  the  signal-to-noise  ratio 


transmitted 

signal 


£  8<t-t0) 
received 
signal 


A 


8x(t) 


.  Kig.  5 


output  of 
matched  filter 
for  input 

a  s (t— tQ) 


Radar  Receiver  as  a  Matched  Filter.  (Radar  signals  are  modulated  carriers. 

*  I  | 

The  envelope  waveshapes  are  shown  Here.) 


21Ref.  [1]  pagi  247  ‘  , 

Ret.  [1]  page  232  .  The  value  of  noise  power  per  unit  bandwidth  as  used  in  the 
present  pap’r  is  twice  as  large  as  it  would  be  if  we  considered  negative  fre¬ 
quencies  as  distinct  from'  positive  frequencies.  The  term  bandwidth  as  we  have 
used  it  refers  to 'positive  frequencies  only. 
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Fig.  6 

Display  of  Received  Radar  Signal  in  Background  of  Noise.  (Pulse  leaves 
transmitter  at  t  ■  0.)  The  pulse  at  tQ  has  Che  waveshape  of  the  auto¬ 
correlation  function  of  the  transmitted  pulse. 


in  the  output  will  have  a  maximum  at  tQ,  and  its  value  there  will  be 


[S(w)]  dw 


2  Total  energy  of  a  s(t)  ^ 

Noise  power  per  unit  bandwidth 


Figure  6  shows  the  display  of  a  received  radar  signal 

Since  a  radar  transmitter  is  usually  limited  in  peak  power,  the  transmitted 
pulse  shape  of  a  long  range  radar  Is  usually  relatively  long  so  as  to  get  a  lot 
of  energy  into  the  pulse.  It  is  also  common  to  put  a  great  deal  of  wave  shape 
detail  in  the  pulse  so  that  among  other  reasons,  it  will  include  considerable  high 
frequency  energy  in  its  waveshape  in  order  to  get  a  sharp  autocorrelation  function 

24 

The  purpose  of  the  matched  filter  Is  to  organize  the  received  signal  so 


that  it  gives  the  maximum  probability  of  detec  ion  of  the  existence  of  a  target. 
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Since  a  radar  display  shows  the  envelope  of  a  modulated  carrier,  the  noise  in 
Fig.  6  is  the  envelope  of  white  gaussian  noise  and  has  the  probability  density 
indicated  in  Eq.  (9) 

24 

The  received  signal  is  the  raw  material  which  is  organized  by  the  matched  filter 
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and  che  greatest  accuracy  in  determining  its  range.  The  frequency  components 
of  the  received  signal  are  the  system  elements  which  are  organized  by  the  matched 
filter.  These  are  constrained  by  the  matched  filter  so  that  they  all  have  zero 
phase  (have  maximum  value)  at  time  This  gives  the  maximum  possible  signal 

amplitude  at  t^  for  a  given  signal  energy.  The  constraint  on  the  frequency 
components  of  both  the  signal  and  noise  imposed  by  the  relation 

|Y(w)  |  -  | S (to)  |  (26) 

coupled  with  the  phase  constraint,  gives  the  maximum  possible  signal-to-noise 
ratio  (on  Che  ensemble  average).  In  other  words,  the  matched  filter  gives  the 
roost  effective  possible  organization  of  the  signal  for  the  purpose  of  target 
detection  and  accuracy  of  range  determination. 

In  che  natural  world  it  takes  energy  to  do  the  things  which  it  is  desired 
to  accomplish.  To  make  this  energy  effective,  it  must  be  organized  by  a  trans¬ 
ducer,  a-'ch  as  the  matched  filter  in  the  foregoing  example.  In  order  to  design 
the  proper  transducer,  one  requires  detailed  information  about  the  elements  of 
the  energy,  i.e.,  analogous  to  the  waveshape  or  magnitudes  and  phaces  of  the 
frequency  components  in  the  matched  filter  case. 

In  thermodynamics,  detailed  information  of  the  energy  elements  is  necessary 
in  order  to  have  a  transducer  to  make  the  energy  effective  for  the  purposes  at 
hand.  However,  it  really  requires  both  the  information  and  the  existence  of  the 
transducer  to  make  the  energy  "available",  i.e.,  effective  In  thermodynamics, 

2 

entropy  measures  the  lack  of  information  about  the  states  of  the  energy  elements 
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It  would  be  more  accurate  to  say  the  Fourier  series  components,  since  the  matched 
filter  is  usually  matched  to  the  waveshape  of  a  single  pulse  as  shown  in  Fig.  5 
and  is  not  matched  to  the  chain  of  repeated  pulses  transmitted  by  the  radar. 

26 

Ref.  [5]  page  160  states  that  "entropy  measures  the  lack  of  information  about 
the  actual  structures  of  a  physical  system." 
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9.  Information,  Organization  and  Thermodynamics 

27 

According  to  information  theory  ,  the  amount  of  information  in  a  message 
concerning  an  event  is 

P2 

log  (27) 

P1 

where  p ^  is  the  probability  of  the  event  before  the  message  is  received,  and  p0 
is  the  probability  of  the  event  after  the  message  is  received.  For  example,  sup¬ 
pose  that  an  observation  is  made  of  the  state  of  a  molecule,  and  the  observation 
indicates  that  the  molecule  is  in  state  A.  Suppose  also  that  the  molecule  is 
actually  in  state  B,  which  may  or  may  not  be  the  same  as  A.  Then  p^  in  Eq.  (27) 
was  the  probability  that  the  molecule  was  in  state  B,  prior  to  the  observation 

and  p.(  is  the  probability  that  the  molecule  is  in  state  B  after  the  observation 
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is  made  ' .  If  there  is  no  noise  in  the  observation  system,  so  that  all  observa¬ 
tions  are  known  to  be  correct,  then  P2  *  1  and  the  amount  of  information  in  the 
message,  according  to  Eq.  (27),  is 

-log  Pj^  (28) 

If  the  possible  results  of  an  observation  are  a  set  {i }  and  if  the  observation 
is  noiseless,  then  the  ensemble  average  of  the  amount  of  Information  in  an  obser¬ 
vation  of  this  kind  is  clearly 


-22- 


Pilog  p± 


(29) 


which  is  identical  with  the  entropy  of  the  set  of  probabilities  of  the  possible 
results  of  the  observation. 

Whether  noise  is  present  or  not,  the  amount  of  information  Indicated  by 
(27)  cannot  exceed  that  given  by  (28).  It  is  also  noteworthy  that  when  the 


^Ref.  fi]  page  4 
28 

We  note  that  p.  and  p,  in  Eq,  (27)  are  the  £  priori  and  a  posteriori  probabilities 
of  the  event  which  actually  takes  place  in  the  world,  regardless  of  whether  or 
not  the  message  reports  the  event  correctly. 


message  is  wrong,  Che  amount  of  information  in  the  message  is  negative.  The 
ensemble  average  of  the  information  per  message,  however,  cannot  be  negative, 
regardless  of  how  noisy  the  observation  system  may  be. 

Any  noiseless  observation  reduces  the  entropy  of  the  set  of  probabilities 
of  the  possible  results  to  zero.  The  observation,  therefore,  represents  a  negen- 
tropy  of  amount  (29).  Information,  entropy  and  negentropy  are  expressible  in  the 
same  units.  The  unit  is  physically  dimensionless,  but  depends  upon  the  base  used 
for  the  logarithms.  When  log,  is  used,  the  unit  is  called  a  bit.  In  engineering 
and  chemical  thermodynamics,  entropy  is  usually  expressed  in  units  of  temper°^ure  • 
One  unic  of  entropy  in  cgs  degrees  Kelvin  units  is  ^  bits,  where  k  -  1.38  x  10  ^ 
ergs/degree.  The  numerical  value  of  the  entropy  or  negentropy  representing  the 
disorder  or  order,  respectively,  of  the  molecules  in  macroscopic  physical  systems 
is  thus  orders  of  magnitude  larger  than  quantities  measuring  information  or  or¬ 
ganization  in  man-made  systems. 

Brillouin  [5]  has  dealt  with  relations  between  information,  negentropy  and 
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physical  entropy  in  detail  .  The  following  conclusions,  taken  from  his  book  , 

are  pertinent  to  our  work: 

(A)  Information  can  be  changed  into  negentropy,  and  vice  versa.  If  the 
transformation  is  reversible,  there  is  no  loss,  but  an  irreversible  transformation 
always  corresponds  to  a  loss. 

(b)  Any  experiment  by  which  information  is  obtained  about  a  physical  system 
produces,  on  the  average,  an  incraase  of  entropy  in  the  system  or  in  its  surround¬ 
ings.  This  average  increase  is  always  larger  than  (or  equal  to)  the  amount  of 
information  obtained.  In  other  words,  information  must  always  be  paid  for  in 
physical  negentropy,  the  price  paid  being  larger  than  (or  equal  to)  the  amount  of 
information  received. 


Claude  Shannon  [7],  [8],  [9]  is  the  architect  of  modern  information  theory.  His 
work  is  fundamental  to  that  in  many  of  the  references  we  have  quoted  and  to  this 
paper.  The  realization  that  entropy  is  an  expression  of  missing  information  goes 
all  the  way  back  to  Boltzmann  according  to  Brillouin  [5]  and  Woodward  [10], 

^Slightly  paraphrased  from  page  184. 
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An  observation  is  always  accompanied  by  an  increase  in  entropy,  and  thus  involves 
an  irreversible  process. 

(C)  The  smallest  possible  amount  of  negentropy  required  in  an  observation 
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is  one  bit 

The  foregoing  discussion  applies  to  available  and  unavailable  energy*  in 
thermodynamics.  An  ideal  gas  in  thermal  equilibrium,  all  at  a  given  temperature, 
is  a  system  in  a  condition  of  maximum  entropy.  In  our  terminology  it  is  com¬ 
pletely  disorganized.  Its  energy  consists  of  the  kinetic  energies  of  its  mole¬ 
cules.  If  there  are  N  molecules  in  the  gas,  then  there  are  3N  independent  system 
elements  in  the  system;  namely,  the  x,  y  and  z  components  of  the  velocities  of 
every  molecule.  The  probability  density  of  the  amplitude  of  each  system  element 

is  gaussian,  for  example,  for  each  molecule 

2 

-rav 

x 


(30) 
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Since  the  velocity  components  of  the  different  molecules,  es  well  as  the  velocity 

components  of  the  same  molecule,  are  all  stochastically  independent  and  are  all 

2 

gaussian  with  the  same  o  ,  the  system  is  of  the  form  of  white  gaussian  noise. 

It  is  therefore  in  a  condition  of  maximum  entropy,  as  already  stated.  The  gas 
has  no  available,  i.e.,  organized,  energy.  If  there  were  any  organized  energy 
present,  it  could,  in  principle  at  least,  be  converted  into  whatever  form  is 
desired,  provided  that  the  nature  of  the  organization  were  known  and  that  a 
suitable  transducer  were  at  hand.  It  is,  therefore,  reasonable  to  say  chat  or¬ 
ganized  energy  is  "available  energy". 

Next,  suppose  that  the  ideal  gas  consists  of  two  separated  parts,  the  first 
part  at  temperature  and  having  molecules,  while  the  second  part  is  at 
temperature  and  consists  of  molecules.  The  gas  now  does  not  represent 
white  gaussian  noise,  since  the  amplitude  distributions  of  3N^  system  elements 
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Briilouin  attributes  this  result  to  Szilard  [6]. 
^Ref.  [11]  page  83.- 
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have  a  a  of  — —  ,  while  the  3N.  have  a  o  of  -  .  This  system  is  therefore 

m  2  m 

not  at  maximum  entropy  and  it  does  have  some  organization  i.a.,  negentropy.  The 
numerical  value  of  the  negentropy  is,  using  Eq.  (14), 


3(NX  +  N2)  log! 


-  31^  log 


-  .  N.T,  +  N.T, 

2tek  22  11 

m  Ni  +  N2 


-  3N2  log 


2fiekT, 


For  example  if 


N  a  M  a  — 
1  2  2 


while 


then  the  negentropy  is 


logf1  2  log  Tl  -  f  -  log  T2  *  |^|log  350  -  |  log  400  -  £  log  30oJ 

p|o. 01487^  bits.  (32) 


If  we  rewrite  Eq.  (32)  in  the  form 


T  +  T 

2nek  1  2  3N  ,  , 

- 2 - —  l0g 


2,ekTi  3N 

- - r  log 


'2itekT, 


*  T  H 


T  -  T 

2nek  1  2 


we  see  that  the  available  energy  is 


3N  k  /T1  "  T2 


since  the  negentropy  is  numerically  equal  to  the  amount  of  entropy  that  would  be 
generated  if  the  available  energy  were  transformed  into  heat.  Equation  (34) 
agrees  with  the  classical  result  obtained  in  thermodynamics. 
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Lec  us  next  investigate  why  the  thermal  energy  of  an  ideal  gas  which  is  all 

at  the  same  temperature  cannot  be  converted  Into  useful  form.  In  order  to  design 

a  transducer  that  would  convert  the  unorganized  thermal  energy  into  a  desired 

organized  form,  it  is  necessary  to  know  the  position  and  velocity  of  every 

molecule.  To  get  this  information  according  to  the  above  quotations  from  Bril- 

louin,  we  would  use  up  more  negentropy  than  the  entropy  of  the  gas.  Since  this 

operation  is  all  taking  place  at  the  same  temperature,  we  would  use  up  more  or- 
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ganized  energy  in  obtaining  the  information  than  we  could  convert  from  thermal 
energy  into  organized  form.  The  thermal  energy  of  an  ideal  gas  which  is  all  at 
the  same  temperature  is,  therefore,  properly  called  unavailable. 

The  situation  is  different  when  the  gas  is  divided  into  two  parts  at  dif¬ 
ferent  temperatures.  In  the  latter  case,  there  is  available  energy.  In  order 
to  know  what  transducer  is  necessary  to  convert  this  available  energy  into  useful 
form,  the  information  about  the  state  of  the  system  which  is  required  is  a  measure¬ 
ment  of  the  respective  temperatures  of  the  two  parts.  The  energy  required  to  do 
this  is  very  small  in  comparison  with  the  energy  which  is  available  as  a  conse¬ 
quence  of  the  temperature  difference.  Available  energy  can  be  tu.ned  into  use¬ 
ful  energy  only  when  the  information  describing  the  organization  which  makes  the 
energy  available  has  (much)  less  negentropy  than  the  physical  negentropy  corres¬ 
ponding  to  the  available  energy. 

10 .  Significance  of  E/kT  and  Woodward's  R 

34 

It  is  shown  in  statistical  mechanics  that  a  small  system  which  is  in  ther¬ 
mal  equilibrium,  with  (or  is  part  of)  a  large  system  at  temperature  T  will  have  a 
probability  of  having  an  energy  given  by  the  formula 


We  note  that  once  the  information  is  obtained,  the  thermal  energy  may  be  considered 
completely  organized.  While  it  would  then  be  possible,  •'heoretically ,  to  build  a 
transducer  to  convert  the  energy  into  any  desired  form,  the  practical  construction 
of  such  a  transducer  is  still  completely  out  of  the  question  Even  if  it  could  be 
built,  however,  the  energy  that  couJd  be  obtained  from  it  is  less  than  the  energy 
used  in  determining  the  position  and  velocity  of  every  molecule. 

"^Ref.  [11]  p.  79 
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-E  /kT 

P(En)  -  Ae  n  (35) 

This  is  the  famous  canonical  distribution  of  Gibbs.  Equation  (35)  applies  to 
quantized  systems.  The  corresponding  formula  for  classical  systems  is 

p(E)dF.  -  Be~E/kTdE  (36) 

One  of  the  methods  of  showing  that  the  thermal  motion  of  a  classical  ideal  gas  has 
the  mathematical  characteristics  of  white  gaussian  noise  is  based  on  Eq.  (36).  An 
interesting  thing  about  Equation  (36)  is  that  it  holds  regardless  of  the  number  of 
degrees  of  freedom  in  the  small  system,  and  regardless  of  the  nature  of  the  energy 
(electrical  or  mechanical,  kinetic  or  potential,  etc.)  so  long  as  there  is  thermal 
equilibrium.  Equations  (35)  or  (36)  apply  to  all  physical  systems  in  thermal  equi¬ 
librium  and  are  not  limited  to  ideal  gases. 

The  derivation  of  Eqs,  (35)  and  (36)  is  based  in  part  on  the  physical  fact 

that  the  probability  density  of  states  for  a  system  in  thermal  equilibrium  is  a 

35  36 

function  only  of  the  energy  This  is  true  both  in  classical  and  quantum 
mechanics.  An  alternative  derivation  of  the  distribution  function  for  the  thermal 
motion  of  an  ideal  gas  could  be  based  on  the  central  limit  theorem  and  the  break¬ 
down  of  constraints  between  system  elements  in  the  stochastic  interaction  between 
loosely  coupled  systems.  We  hope  later  to  use  this  to  gain  insight  into  the  in¬ 
trinsic  nat "rn  of  energy. 

E 

The  analogue  of  n/kT  in  an  information  transmission  system  is 

R  m  _ Total  signal  energy _  ,^.37 

2  Noise  power  per  unit  bandwidth  > 

33Ref.  Ill]  p.  12 

36Ref .  (11]  p.  20 
37 

Referring  co  Eq.  (30)  we  see  that  the  analogue  of  the  mean  noise  energy  per  system 
element  is  kT/2,  Taking  into  account  that  there  are  independent  in-phase  and  qua¬ 
drature  elements  at  a  given  frequency,  the  analogue  of  noise  power  per  unit  band¬ 
width  is  kT,  The  temperature  of  a  physical  system  is  thus  the  analogue  of  noise 
power  per  unit  bandwidth. 
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The  symbol  R  was  introduced  by  P.  M.  Woodward  ,  who  first  showed  its 

pivotal  importance  in  information  transmission.  Comparing  Equations  (37)  and 

(23) ,  we  see  that  R  is  the  signal-to-noise  energy  ratio  at  the  output  of  a  matched 
En 

filter.  R,  like  ,  is,  except  for  a  normalization  constant,  proportional  to  the 

logarithm  of  the  probability  that  the  signal  (or  system  organization)  is  a  noise 

39 

fluctuation,  regardless  of  the  signal  waveshape 
R  E 

—  and  n/kT  are  completely  analogous  only  if  En  is  a  single  organized  (quantum) 
40 

state  .  If  the  energy  is  transformed  into  thermal  energy  at  the  temperature  T, 

g 

this  creates  an  increase  in  entropy  of  amount  n/kT.  Therefore,  the  neRentropy  of 

E 

the  organized  state  is  n/kT.  On  the  other  hand,  the  negentropy  represented  by  R 
is  numerically  equal  to  the  maximum  entropy  that  could  be  generated  by  transforming 
the  signal  energy  into  white  gausslan  noise.  If  2TB  is  the  number  o  sample  values 
which  are  available  in  which  to  express  the  signal,  then  according  to  Eq.  (17),  the 
negentropy  represented  by  R  is 

18  108  ( IS  +  l)  l38) 

If  P  is  less  than  or  equal  to  2TB,  then  the  negentropy  is  very  nearly 
^  approximately 

R  .  Thus,  R  is/the  maximum  negentropy  possible  for  the  signal  which  has  a  given 
2  2 

ratio  of  total  signal  energy  to  noise  power  per  unit  bandwidth,  and  it  can  only 
reached  if  there  are  at  least  R  sample  values  available  in  which  to  express  the 
signal. 

We  have  already  noted  that  white  gaussian  noise  has  a  statistically  indepen- 

2 

dent  gaussian  distribution  of  the  same  ^  in  any  orthonormal  basis  system  of  vec¬ 
tors  in  function  space  Therefore,  if  we  U3e  the  wave  shape  of  the  desired  signal 

Ref.  (10]  pages  85-80  and  102-114. 

J  Kef .  [12]  p,  589. 

40 

In  the  classical  case,  if  all  the  energy  E  were  in  one  degree  of  freedom,  then 
E/kT  would  also  be  analogous  to  R/2. 

“^When  we  discuss  the  ambiguity  threshold  in  the  next  section,  we  will  show  that 
the  organized  signal  will,  in  general,  become  disorganized  if  it  is  spread  over 
a  number  of  sample  values  greater  than  R 


2X 
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as  one  basis  vector  and  if  we  use  a  filter  which  passes  only  this  waveshape  of  the 

entire  orthonormal  9et,  the  output  of  the  filter  will  have  a  aignal-to-noise  ratio 

given  by  R,  Thus  R  is  the  signal-to-noiae  ratio  if  all  the  signal  energy  has  been 
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put  into  one  degree  of  freedom.  It  may  be  shown  that  this  is  what  is  done  by 
the  matched  filter. 

Noise  Reduction  and  Thresholds  in  Communication 

A  topic  which  is  intimately  related  to  the  organization  of  signals  is  the 

question  of  noise  reducing  methods  of  organization  in  communication  systems. 

Human  Beings,  without  consciously  being  aware  of  the  fact,  use  sophisticated 

methods  of  noise  reduction  in  both  audio  and  visual  perception.  The  first  well- 

conceived  idea  of  a  noise  reducing  system  in  communication,  however,  is  probably 

43 

the  invention  by  E.  H<  Armstrong  of  wideband  frequency  modulation  for  the  purpose 
of  noise  reduction. 


Detailed  discussion  of  noise  reduction  in  frequency  modulation  is  given  in 
44 

various  textbooks  but  because  of  the  importance  of  the  method  as  an  illustration 
of  a  fundamental  phenomenon  of  organization,  we  shall  outline  its  details  here. 

A  signal 

G(t)  A  sin  1 2 tt F t  +  2ir  \s(t)dt]  (39) 

has  an  Instantaneous  frequency 

*57^  (2ttFc  +  2*  fs(t)dt)  -  F  +  s(t) 

The  signal  G(t)  is  called  a  frequency  modulated  carrier ,  where  F  i9  the  carrier 
f req uency  and  s(t)  is  the  modulation  signal .  In  order  to  understand  the  mechanism 
of  F.M.  noise  reduction,  we  first  consider  the  case  when  s(t)  is  the  single  four- 
ier  component 


innt^ass 
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c„  ( t)  «  A  F  cos  2itMt 
45 

It  may  then  be  shovm  that 


(41) 


G^(t)  -  A  sin[2ffFt  +  sin  2ngt] 

■  a-ijo  |n sin  2'rt 


AF 


sin  [2n(F  +  p)t]  -  |  —  /  sin  1 2  t.  ( F  -  w)t] 


+  J2  '\~J  sin  [2n(F  +  2g)  +  J2  sin  [2n(F  -  2p)tj 


(42) 
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AF 


The  components  of  the  fora  +  j  sin  (2n(F  +  np)tj  are  called  sidebands  of  the 

carrier  sin  2nFt,  g  is  called  the  modulation  frequency,  the  coefficients 
are  called  the  magnitudes  of  the  sidebands,  and  AF  is  called  the  peak  frequency 
deviation.  In  Fig.  7  is  a  diagram  showing  the  sideband  magnitudes  in  a  typical 
case  of  wideband  frequency  modulation.  The  sideband  magnitudes  quickly  become 
negligible  outside  che  range  (F  +  AF) .  It  is  therefore  customary  to  consider  that 
there  are  approximately  2  effective  sidebands.  When 

(43) 


we  are  dealing  with  what  is  called  wideband  frequency  modulation. 


*F  -  75  kc. 
<1,  -  3.75  kc. 


Sideband  magnitudes  illustrating  Equation  (42) 


45Ref.  [14]  pages  149-151. 

4^J  /— )  is  the  Bessel  function  of  the  first  kind  of  order  n  and  argument  — 

n  I  u  /  u 
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Ref.  (14)  pages  272-273. 
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Suppose  that  white  gausaian  noise  is  superimposed  on  a  wideband  frequency 
modulated  signal  G(t)  of  the  form  (39)  and  the  signal  is  sent  through  a  trans¬ 
ducer,  called  a  frequency  detector,  whose  output  is  s(t)  plus  noise.  It  may 
be  shown^  that  in  this  case  the  signal-to-noise  ratio  is  improved  as  compared 
with  what  would  be  obtained  by  an  amplitude  modulation  system  using  the  same 
transmitted  signal  power  and  having  the  same  superimposed  white  gausaian  noise. 
In  wideband  F.M. ,  the  signal-to-noise  a.mplitude  ratio  is  improved  by  a  factor 

At?  /  .  f.  \  2 


proportional  to  —  ,  i.e.,  proportional  to 


(r) 


in  power  or  energy.  The  de¬ 


tails  of  the  calculation  may  be  found  in  the  cited  reference,  but  we  will  now 
examine  what  makes  this  noi9e  improvement  possible  from  the  point-of-view  of 
organization  theory. 

In  the  first  place  it  should  be  pointed  out  that  an  improvement  in  signal-to- 
noise  ratio  is  only  significant  to  the  extent  that  it  signifies  an  increase  in  the 
amount  of  information  received.  For  example,  suppose  that  a  signal  a(t)  ha9  super¬ 
imposed  white  gausaian  noise  and  the  signal-to-noise  ratio  is  3.  If  this  combina¬ 
tion  is  sent  through  a  device  whose  output  is  proportional  to  the  cube  of  the  input, 
the  signal-to-noise  ratio  in  the  output  would  be  of  the  order  of  magnitude  of  27. 
However,  even  though  the  signal-to-noise  ratio  is  increased,  the  information  is 
unchanged.  The  probability  distributions  of  the  signal  and  noise  amplitudes  have 
been  changed  and  the  noise  is  no  longer  gaussian,  but  the  negentropy  as  well  as  the 
information  content  are  unchanged.  The  signal  waveshape  is  distorted  by  the  cubic- 
law  amplifier,  and  there  is  no  way  of  using  the  increased  signal-to-noise  ratio  to 
regain  the  signal  with  undistorted  waveshape  but  with  an  improved  signal-to-noise 
ratio. 

The  improvement  of  signal-to-noise  ratio  in  wideband  frequency  modulation  is, 
however,  informationally  significant.  It  can  be  obtained  without  changing  the  wave¬ 
shape  of  the  desired  signal  and  with  a  lower  level  of  background  noise  in  the 
output . 


Ref,  [14]  pages  272-273. 
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Let  us  compare  amplitude  versus  frequency  modulation  for  the  case  in  which 
the  modulation  signal  s(t)  has  a  bandwidth  ug  and  the  peak  frequency  deviation 
available  for  the  frequency  modulated  carrier  is  AF.  Then  the  effective  band¬ 
width  of  an  analogous  amplitude  modulated  signal  ia  2uq,  whereas  the  bandwidth 
of  the  frequency  modulated  signal  is  essentially  2Af.  Let  us  suppose  that  the 
total  signal  energy  and  the  noise  power  per  unit  of  bandwidth  is  the  same  in 
both  cases,  so  that  Woodward's  R  is  the  same  for  both.  Then  by  Eq.  (38)  the 
negentropy  represented  by  a  signal  with  the  bandwidth  of  the  amplitude  modulated 
signal  is  at  most 


108  fjfr + 1" 

_  O 


(44) 


whereas  a  signal  with  the  bandwidth  of  the  frequency  modulated  signal  could  possibly 
have  the  larger  negentropy 


2TAF  log 


pL_+  il 

[4TAF  _ 


(45) 


assuming  that  R>4TAF.  The  actual  negentropy  in  the  two  cases  is  not  given  by 
Eqa.  (44)  and  (45)  but  depends  on  the  probability  distributions  and  the  constraints 
in  the  respective  cases. 

A  comparison  of  (44)  and  (45)  indicates  that  a  type  of  modulation  might  exist 

S  1 

in  which,  for  large  R,  the  equivalent  —  power  ratio,  without  waveshape  distortion, 

would  Increase  almost  exponentially  with  the  ratio  of  increase  in  bandwidth  (such 
AF 

as  —  ) .  We  are  not,  however,  interested  in  the  properties  of  frequency  modulation 
o 

per  se,  but  in  the  fact  that  the  organization  of  a  signal  can  be  improved  by  using 
extra  bandwidth  in  transmission.  It  is  interesting  to  note  that  a  simple  repetition 
of  the  signal  N  times  with  the  same  total  energy,  so  that  the  energy  each  time  is 
reduced  by  the  factor  ^  ,  would  give  no  net  gain  in  negentropy  and  no  improvement 
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s  s 

in  —  ratio.  Systems  which  do  give  q  improvement  by  using  axtra  bandwidth  (or 
extra  time)  without  using  more  energy  are  somehow  able  to  translate  their  re- 

I 

dundant  representation  of  the  signal  into  an  improved  organization.  Let  us 
examine  how  this  is  accomplished  in  wideband  frequency  modulation. 

The  transducer  which  translates  the  modulated  carrier  G(t)  In  Eq.  (39)  into 
the  signal  s(t)  is  called  a  frequency  detector. 


If  the  signal  G(t)  plus  the  noise  n(t)  is  represented  as 


G(t)  +  n(t)  -  B(t)sin[2nFt  +  $(0  3 

I 

Chen  the  output  of  the  frequency  detector  is 

■  1_  d£ 

2 if  dt 


(46) 


(47) 


Part  of  the  mechanism  of  noise  reduction  in  wideband  frequency  modulation  is 
illustrated  in  Fig.  8(a).  For  a  single  modulation  frequency  u  which  causes  a 
maximum  frequency  swing  Af  of  the  carrier,  the  maximum  value  of  6  (see  Eq.  42) 
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Is  radians,  which  car.  ba  as  large  as  many  complete  revolutions.  On  the  other 
hand,  if  ■^■^>>1,  then  the  effect  of  the  noise  is  to  cause  a  change  in  the  angle 
of  amount 

-1  N' 


-  0 


tan 


(48) 


which  is  small  compared  with  The  signal  is  thus  more  efficient  than  the  noise 

in  frequency-modulating  the  carrier.  This  effect  is  the  first  part  of  the  noise 
reduction  mechanism  in  wideband  F.M,  The  second  part  is  obtained  by  filtering. 


If 


4TAF 


is  considerably  greater  than  unity,  then  only  those  noise  components  in 


the  frequency  range  (F  +  )  will  cause  noise  modulation  in  the  modulation  fre¬ 

quency  range.  The  frequency  detecting  transducer  includes  a  filter  which  will 
not  pass  any  frequencies  outside  the  modulation  frequency  range.  This  filter,  and 
the  fact  that  it  can  eliminate  the  affects  of  noise  components  outside  the  frequency 
range  (F  +  pq)  is  the  second  part  of  the  noise  reduction  mechanism  of  wideband 
F.M.48 

There  is,  however,  a  threshold  in  F.M.  noise  reduction.  This  is  illustrated 

2  2 

in  Fig,  8(b).  If  R<4IAF  so  that  (S')  <  (N*)  ,  then  the  transducer  acts  so  that 

the  noise  reduces  the  signal  instead  of  vice  versa,  and  in  addition  to  reduction 

in  magnitude  the  signal  becomes  distorted.  These  effects  become  exaggerated  as 

_ R_ 

4TAF 


gets  smaller.  The  region  in  the  neighborhood  of 


4taT 


-  1 


(49) 


is  thus  a  threshold  between  noise  elimination  and  signal  elimination.  When 
AF 

— *>>1,  it  is  a  sharp  threshold. 
o 

The  existence  of  a  noise  reduction  threshold  is  a  general  phenomenon  in 
virtually  all  communication  systems  which  use  extra  bandwidth  or  extra  time  for 


48 


See  Ref.  [14]  p.  272-273  for  a  quantitative  discussion. 
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noise  reduction  purposes,  without  using  extra  energy  .  In  the  next  section, 
we  shall  see  that  an  analogous  threshold  is  very  important  in  other  organized 
systems.  It  was  first  pointed  out  by  Shannon  (9]  that  a  noise  reduction  thres¬ 
hold  phenomenon  could  be  expected  in  the  transformation  of  the  representation  of 
a  signal  from  a  function  space  of  higher  dimensionality,  say  4TAF,  to  a  smaller 
dimensional  (say  2Tuq)  function  space.  A  similar  point  was  made  by  Woodward^ 
during  his  classical  discussion  of  noise  thresholds  in  radar  detection.  A 
function  space  analysis  of  noise  reduction  and  threshold  for  a  large  class  of 
communication  systems  is  given  by  Wozencraft  and  Jacobs^. 

The  function  space  analysis  shows,  in  effect,  that  in  the  transformation 
or  mapping  from  the  4TAF  dimensional  space  at  the  input  of  the  transducer  to 
the  2Tuq  dimensional  space  at  the  output,  some  of  the  white  gaussian  amplitude 
noise  at  the  input  is  transformed  into  identity  noise  in  the  output.  This 

transformation  of  amplitude  noise  into  identity  noise  is  dependent  upon  ^ 

R  52 

and  rises  suddenly  from  a  small  to  a  large  value  when  falls  below  unity  . 

This  is  the  noise  reduction  threshold  phenomenon. 

It  is  of  interest  to  note  that  when  ■.  ■  -  falls  below  unity,  there  is  on 

4TAF 

the  average  less  than  one  bit  of  information  per  system  element  in  the  input 
signal  to  the  transducer.  As  a  consequence  of  item  C  in  the  quotation  frt>m 
Brillouin  in  Section  9,  we  would  therefore  expect  chat  the  information  could  not 
be  extracted.  This  is  quite  consistent  with  the  existence  of  the  noise  reduction 
threshold. 


49 

See  Ref.  [4]  p.  623  et  seq  for  a  discussion  of  noise  reduction  and  threshold 
in  pulse-position  modulation. 

50Ref.  [10]  p,  90. 

''"Ref.  (4]  Ch.  9 
52 

Lance  Whitehead  (unpublished)  has  shown  that  when  the  input  dimensionality  is 
high,  the  threshold  is  sharp.  This  is  a  consequence  of  the  mathematical 
phenomenon  that  almosc  all  of  the  volume  of  a  hypersphere  of  high  dimensionality 
is  close  to  the  surface. 
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The  phenomena  of  noise  reduction  and  thresholds  can  also  be  considered  from 
a  more  elementary  point  of  view  in  which  emphasis  is  placed  upon  the  mechanism 
of  the  noise  reduction.  In  an  early  paper  the  present  writer  has  said  "When 
the  noise  exceeds  the  improvement  threshold  value,  the  signal  no  longer  controls 
the  standard  by  which  coherence  is  determined  and  on  the  basis  of  which  the 
detector  (transducer)  is  designed  to  give  preferred  weighting  to  the  desired 
signal.  Under  these  circumstances,  the  detector  increases  the  noise-to-slgnal 
ratio  rather  than  the  signal-to-noise  ratio."  In  Fig.  8(b),  we  have  seen  how 
control  of  the  standard  of  coherence  in  frequency  modulation  passes  from  the 
signal  to  the  noise  at  the  noise  improvement  threshold. 

A  detailed  examination  of  systems  which  use  extra  bandwidth  for  noise  re¬ 
duction  purposes  seems  always  to  show  a  standard  of  coherence  which  is  lost 
when  the  noise  exceeds  the  Improvement  threshold.  When  the  signal  controls  the 
standard  of  coherence,  the  negentropy  of  redundant  signal  in  the  extra  bandwidth 
is  efficiently  converted  by  the  transducer  into  negentropy  of  desired  signal. 

When  the  standard  of  coherence  is  lost,  the  signal  organization  collapses. 

It  is  particularly  noteworthy  that  a  system  which  uses  extra  bandwidth  for 
noise  reduction  actually  Increases  the  effective  R  of  the  modulation  s(t)  in 
the  output  signal.  By  way  of  contrast,  we  note  that  the  matched  filter  does  not 
increase  the  effective  R.  The  matched  filter  also  does  not  have  a  noise  reduction 
threshold.  The  action  of  the  matched  filter  is  equivalent  to  a  linear  operation 
in  function  space  that  transforms  the  representation  of  the  signal  plus  noise 
onto  a  single  system  element  where  the  signal  waveshape  is  Che  basis  vector. 
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Ref.  [12]  p.  592. 
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12.  THE  SELF-MAINTENANCE  OF  AN  ORGANIZED  ACTIVITY  BY 
USING  NEGENTROPY  ASSIMILATED  FROM  THE  ENVIRONMENT 

(a)  Introduction 

A  moat  important  class  of  organized  systems  are  those  which  include  an  or¬ 
ganized  activity  that  assimilates  negentropy  from  its  environment  (i.e.,  from 
external  sources)  in  order  to  maintain  itself.  A  running  automobile  motor  and 
a  biological  organism  would  be  examples  of  such  systems.  In  these  systems  the 
organized  activity  is  constantly  being  attenuated  by 

(a)  Transformation  into  heat,  or 

(b)  Absorption  in  a  work  load,  or 

(c)  Radiation  and  its  analogues. 

At  the  same  time,  the  activity  accumulates  noise  both  from  its  internal  operations 
and  from  the  environment.  Negentropy  must  be  assimilated  both  to  replace  the 
attenuation  losses  and  to  perform  the  necessary  operations  to  reduca  the  noise. 

We  shall  introduce  our  analysis  with  the  aid  of  some  illustrative  examples. 

We  shall  not  discuss  how  each  point  that  we  make  is  Illustrated  in  every  example, 
but  we  hope  that  the  reader  will  in  each  case  check  aspects  of  the  examples. 

The  first  five  examples  represent  activities  which  are  reasonably  stable  over 
periods  of  time  of  interest.  The  sixth  example  illustrates  the  maintenance  of 
a  stable  activity  while  travelling  in  space. 

(b)  Examples 

1.  The  Clock 

We  shall  consider  the  case  whether  the  organized  activity  is  either  an 
oscillating  pendulum  or  an  oscillating  balance  wheel.  In  either  case,  when 
the  oscillation  reaches  a  certain  critical  magnitude  or  threshold,  it 
operates  an  escapement  to  release  some  energy  from  a  lifted  weight  or 
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a  wound  spring  to  add  Co  the  anergy  of  oscillation.  This  Jusc  cancels  the 
attenuation  of  the  oscillation  due  to  the  energy  lost  in  controlling  the 
motion  of  the  clock. 

2.  The  Bunsen  Flame 

The  flame  of  a  Bunsen  burner  has  a  characteristic  size,  shape  and 
distribution  of  color  and  temperature,  which  is  a  function  of  the  gas 
pressure,  the  amount  of  air  Intake,  and  the  nozzle  configuration.  The  flame 
is  a  quasi-stationary  activity,  but  the  materials  engaged  in  the  activity 
at  any  Instant  are  quickly  carried  away  by  convection  into  the  surroundings, 
while  the  energy  leaves  by  radiation,  convection  and  conduction.  The  move¬ 
ment  of  the  heated  material  from  the  flame  draws  in  new  gas  and  air  through 
the  Bunsen  burner  as  well  as  air  from  the  surroundings,  and  the  combustion 
of  these  materials  keeps  the  flame  constantly  renewed. 

The  negentropy  is  taken  from  the  surroundings  in  the  form  of  gas 
(say  CH^)  and  oxygen.  This  is  converted  into  the  flame  organization  by  a 
pattern  of  combustion,  convection  and  radiation.  We  note  that  in  order  to 
gat  the  flame  started  a  certain  degree  or  threshold  of  the  flame's  organ¬ 
ization  must  first  be  reached,  such  as  by  putting  a  lighted  match  near  the 
nozzle.  Once  this  degree  of  organization  is  reached,  the  flame  then  proceeds 
toward  its  characteristic  configuration.  If  later  on,  a  gust  of  wind  or 
some  other  disturbance  changes  the  organization  to  something  sufficiently 
different  from  the  characteristic  configuration,  the  flame  will  be  exting¬ 
uished  and  will  no:  start  up  again  without  outside  help. 

3.  The  Running  Automobile  Motor 

We  shall  not  go  into  the  details  of  the  organization  and  operation  of 
Che  automobile  motor.  We  assume  that  the  reader  is  sufficiently  familiar 
with  them  for  the  purposes  at  hand.  In  the  case  of  the  running  automobile 
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motor,  most  of  the  activity  can,  of  course,  be  absorbed  in  a  work  load. 

Just  as  in  the  case  of  the  Bunsen  flame,  in  order  to  get  the  motor 
started,  a  certain  approximation  to  the  organization  of  the  motor's  op¬ 
erating  conditions  must  first  be  established  by  a  starting  routine.  Once 
this  has  been  done,  the  motor  activity  proceeds  to  its  characteristic  quasi- 
stable  organization.  The  motor  will  continue  to  run  without  outside  assist¬ 
ance  as  long  as  supplies  are  available  in  the  form  of  gasoline,  etc.  to 
renew  the  negentropy  which  is  being  used  up.  If,  however,  some  disturbance 
changes  the  configuration  of  the  motor's  activity  to  something  sufficiently 
different  from  its  characteristic  organization,  the  motor  will  stall.  Some 
person  or  external  influence  must  then  carry  through  the  starting  routine 
again  in  order  to  get  the  motor  running  once  more. 

We  also  point  out  that  the  waste  products  of  combustion,  as  well  as 
excess  heat  in  certain  locations,  represent  noise  which  must  be  eliminated 
if  the  organization  is  to  be  maintained.  There  are  also  other  types  of 
noise  which  are  more  involved  with  the  details  of  the  operation. 

4.  The  Electronic  Oscillator 

In  an  ordinary  amplifier,  the  output  is  a  magnified  replica  of  the 
input,  in  a  regenerative  amplifier  the  output  is  fed  back  to  the  input  and 
goes  through  the  amplifier  for  repeated  magnifications.  The  amount  of 
amplification  of  the  amplifier  in  one  passage,  and  the  percentage  of  the 
output  which  is  fed  back  to  the  inputj  as  well  as  the  relative  phase,  are 
usually  all  functions  of  frequency.  It  is  possible  to  build  a  regenerative 
amplifier  so  that  with  no  input  other  than  its  own  internal  noise  of  thermal 
origin,  it  will  have  a  large  output  at  a  chosen  frequency.  This  is  an 
electronic  oscillator.  Its  frequency  is  determined  by  its  circuit  components 


and  connections 
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The  energy  and  negentropy  of  Cha  alactrical  oscillations  must  constantly 
ba  replenished  from  the  power  supplies  of  the  oscillator.  The  oscillator 
differs  from  the  clock,  the  bunsan  flama  and  tha  automobile  motor  in  that 
it  does  not  have  to  bs  star tad  by  first  satablishing  an  approximation  to 
the  organized  activity  by  external  means.  The  components  near  its  char¬ 
acteristic  frequency  in  the  background  noise  are  sufficient  to  start  it. 

5.  Tha  Living  Organism 

We  shall  treat  the  organizational  aspects  of  a  living  organism  at 
greater  length  in  later  sections.  At  this  time  we  just  take  note  of  the 
fact  that  the  activity  of  life  uses  up  negentropy  which  must  be  replaced 
by  extracting  it  from  food,  water  and  air  which  is  taken  in.  Among  many 
other  meuns  of  noise  reduction,  the  living  organism  has  extensive  means 
for  the  elimination  of  waste  products.  Like  the  flams  and  the  automobile 
motor,  the  life  activity  will  be  extinguished  if  its  organization  becomes 
too  far  removed  from  its  characteristic  pattern.  If  the  organism  dias, 
it  cannot  be  brought  back  to  life.  Within  the  range  of  our  experience, 
life  activity  is  always  a  continuation  of  previous  life  activity. 

6.  Long  Distance  Communication  with  Periodic  Repeaters 

In  communication  channels  which  transmit  signals  between  locations 
which  are  thousands  of  miles  apart  on  the  surface  of  the  earth,  it  is 
common  to  use  wliut  are  sometimes  called  repeaters  at  periodic  locations 
between  the  original  signal  transmitter  and  the  final  destination.  Thus, 
when  the  sending  and  receiving  stations  are  2,000  miles  apart,  repeaters  may 
be  located  at  200  mile  intervals  along  the  transmission  path.  In  the  200 
miles  between  repeaters,  the  signal  is  attenuated  and  is  corrupted  by  noise. 
At  the  repeater,  the  noise  is  largely  eliminated  and  tha  signal  is  amplified 
to  restore  its  original  level.  Assuming  that  the  distance  between  repeaters 
has  been  chosen  short  enough  so  thst  the  noise  does  not  rise  about  the  noise 
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reduction  threshold  before  the  repeater  is  reached,  this  allows  the  signal 
coming  out  of  the  repeater  to  be  practically  a  clean  and  faithful  replica 
of  the  original  transmitted  signal.  In  this  way  it  is  possible  to  have 
essentially  noise  free  transmission  over  thousands  of  miles. 

The  particular  noise  reduction  means  illustrated  in  Fig.  9  is  the  use 
of  pulse  frequency  modulation.  In  this  case  the  unmodulated  carrier  consists 
of  a  sequence  of  equally  spaced  pulses,  for  example  pulses  of  1  mocrosecond 
duration  occurring  at  a  rate  of  100,000  pulses  per  second.  Modulation  of  the 
carrier  is  accomplished  by  changing  the  pulse  rate  in  accordance  with  the 
wave  shape  of  the  modulation  signal.  A  large  part  of  the  noise  reduction 
is  accomplished  at  the  repeater  by  means  of  a  device  called  a  limiter  which 
passes  only  the  signal  which  lies  between  the  limits  and  L ^  of  the  input 
signal,  This  signal  is  then  amplified  up  to  the  level  of  the  original 
transmitted  signal.  There  are  several  other  aspects  of  this  noise  reduction 
process  which  we  have  not  discussed,  for  which  we  refer  the  reader  elsewhere 
[14]. 

We  note  that  the  noise  reduction  threshold  is  the  noise  level  at  which 
the  noise  between  pulses  rises  above  L ^  or  the  noise  during  pulses  drives 
the  signal  below  .  This  example  also  illustrates  the  phenomenon  that  a 
quantized  signal,  such  as  a  pulse,  can  be  made  relatively  refractory  to 
the  effects  of  small  amounts  of  noise. 

The  attenuation  of  the  organization  illustrated  in  this  example  is  an 
attenuation  in  space  rather  than  in  time.  It  represents  a  loss  of  negen- 
tropy  which  is  compensated  by  absorbing  negentropy  from  the  power  supplies 
of  the  amplifiers  at  the  repeater  stations.  This  example  is  especially 
instructive  because  it  is  so  completely  understood.  It  illustrates  the 
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compensation  for  attenuation,  the  noise  reduction  and  the  noise  reduction 
threshold  which  are  characteristic  of  most  self-sustaining  activities. 

(c)  Stability 

The  organized  activities  in  the  foregoing  examples  all  exhibited  a 
type  of  stability  which  is  shown  in  Fig.  10.  It  should  be  emphasized  that  we 
are  describing  a  state  of  activity  and  not  a  static  condition.  Thi9  activity 
may  have  periodic  components,  and  the  state  therefore  cannot  be  described  in 
terms  of  an  instant  of  time,  but  requires  at  least  a  time  duration  of  the 
longest  periodic  component.  In  some  representations  of  the  state  diagram 
of  the  system,  the  stable  point  in  Fig.  10  may  be  represented  by  a  closed 
curve. 

Figure  10  is  a  diagrammatic  representation  of  a  state  diagram.  It  shows 
only  a  cross  section  of  a  multidimensional  picture,  since  the  state  of  the  ac¬ 
tivity  typically  has  many  degrees  of  freedom.  A  stable  point  on  the  diagram 
represents  a  stable  operating  state.  Any  small  displacement  of  the  system  from 
the  stable  operating  state  causes  a  reaction  by  the  system  which  tends  to  br^ng 
the  system  hack  to  the  stable  state.  However,  in  any  direction,  there  is  a 
threshold  value  to  the  size  of  the  allowable  displacement.  If  the  displacement 
exceeds  this  threshold,  the  system  will  not  return  to  the  stable  point.  A 
system  which  has  no  such  stable  point  cannot  maintain  itself  in  an  organized 
condition. 

The  existence  of  the  stable  state  implies  that  a  certain  balance  exists 
when  the  system  is  in  this  state,  i.e., 

1.  Available  energy  is  taken  in  from  the  environment 
as  fast  as  it  is  used  up. 

2.  System  negentcopy  is  taken  in  as  fast  as  it  is  used  up. 

3-  Waste  products  are  removed  as  fast  as  they  are  generated. 

4.  Heat  is  removed  as  fast  as  it  is  generated. 
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Small  time  delays  are  allowed  in  these  balancing  operations,  as  long  as  the  dis~ 

placement  fluctuations  from  the  stable  point  caused  by  the  delays  do  not  push 
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the  operating  state  outside  of  threshold  , 

The  state  diagram  in  Fig,  10  is  a  property  of  the  system  constraints.  We 
shall  call  the  state  of  the  system  at  the  stable  point  an  eigenstate  of  the 
system  of  constraints.  Alternatively,  we  may  call  the  entire  region  around  the 
stable  point  up  to  the  thresholds  an  eigenstate  of  the  system  of  constraints. 

As  already  mentioned.  Fig.  10  is  diagrammatic  of  what  is  meant  by  a  stable 
state.  It  implies  that  the  displacement  of  a  system  from  the  stable  point  causes 
a  reaction  by  the  system  which  tends  to  bring  the  system  back  to  the  stable  point. 
There  is  no  ja  priori  guarantee  that  stable  states  of  activity  can  be  produced. 
Experience,  however,  tells  us  that  they  can,  as  illustrated  by  the  examples  in 
Sec.  12b.  Le  Chetalier's  principle  in  thermodynamics  shows  that  for  an  isolated 
system,  the  equilibrium  state  is  stable'*'’. 

A  detailed  discussion  of  a  stable  state  of  a  specific  organized  activity  is 
usually  very  complicated.  However,  it  seems  worthwhile  to  show  in  at  least  one 
particular  case  how  the  stability  is  attained.  We  therefore,  consider  the  op¬ 
eration  of  a  simplified  version  of  an  automobile  motor-  In  Fig.  11,  the  abscissa 
represents  the  horizontal  velocity  of  the  car.  The  ordinate  is  the  torque  avail¬ 
able  from  the  motor  to  turn  the  wheels  of  the  car.  It  is  assumed  that  the  car 
is  moving  along  a  level  road  of  standard  paving  and  that  the  atmospheric  condi¬ 
tions  are  also  standard.  The  solid  curve  in  Fig,  11  is  the  torque  demand  re¬ 
quired  to  maintain  the  car  at  a  given  velocity  against  the  frictional  forces  of 
the  air,  the  road  and  the  motor  and  other  internal  operating  parts-  The  broken 
curve  in  Fig.  11  represents  the  torque  supply  from  the  motor  for  given  settings 
of  the  gears  and  the  accelerator.  The  torque  supply  rises  monotonically  with 
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Prigogene  has  shown  that  in  a  stable  state,  the  rate  of  dissipation  of  free 
energy  is  a  minimum  Ref.  [15]  Ch,  16, 
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Ref.  [11]  page  63. 
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Fig.  11 


Che  velocity  of  Che  car,  because  che  race  of  fuel  Intake  rises  wiCh  Che  velocity 
of  Che  car.  In  particular,  che  race  of  inCake  of  fuel  (gasoline  and  air  mixCure) 
is  proportional  to  cha  number  of  sCrokes  per  minuCe  of  Che  cylinders^  and  che 
number  of  scrokes  per  minute  of  che  cylinders  is  proporcional  Co  Che  veloclcy 
of  che  car  =  For  a  given  setcing  of  che  gears  and  che  accelerator,  che  fuel  in¬ 
cake  is  thus  proporcional  co  Che  velocity  of  che  car.  The  efficiency  of  Che 
combustion  process,  however,  varies  with  che  rate  of  fuel  intake  so  chat  the 
broken  curve  in  Fig.  11  is  noC  a  straight  line. 

If  che  torque  supply  exceeds  che  torque  demand,  Che  car  is  accelerated. 

On  che  other  hand  if  che  Corque  supply  is  less  Chan  che  torque  demand,  the  car 

l 

is  decelerated.  Consequently,  as  long  as  the  velocity  exceeds  V^,  it  will  tend 
Coward  V  ,  which  is  the  velocity  specified  in  che  design  of  the  car  for  the 
given  conditions.  If  Che  velocity  should  fall  below  V  ,  it  will  tend  toward 
zero.  This  will  cut  off  the  fuel  supply  and  the  motor  will  stall. 

From  che  foregoing  discussion,  we  may  draw  the'  following  conclusions.  A 
smoothly  running  automobile  motor  operates  with  positive  feedback  (regenera- 
tively)  in  its  interaction  with  its  nutrient  environment  i.e.  ,  che  fuel  supply. 
Thus,  as  the  activity  increases,  it  interacts  with  the  fuel  supply  to  increase 
the  rate  of  fuel  supply  which  tends  to  increase  the  activity  still  further, 

On  the  other  hand,  operating  in  the  total  environment  for  which  it  is  designed 
including  both  the  fuel  supply  and  the  work  to  be  done,  a  smoothly  running  auto¬ 
mobile  motor  operates  with  negative  feedback  to  stabilize  itself  at  the  operat¬ 
ing  condition,  V  ,  which  corresponds  to  its  design  specifications.  Thus,  if  the 
A 

velocity  is  less  chan  V^,  the  difference  between  torque  supply  and  demand  pro¬ 
vides  acceleration  to  increase  the  velocity.  On  the  other  hand,  if  the  velocity 
exceeds  V  ,  the  difference  changes  sign  so  that  there  is  deceleration  to  diminish 

A 

the  velocity. 

J^The  setting  of  the  accelerator  determines  the  rate  of  fuel  intake  per  cylinder 
stroke- 
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The  point  A  in  Fig.  11  corresponds  to  the  stable  point  in  Fig.  10.  The 

c  7 

Point  B  corresponds  to  a  threshold"'  . 

(d)  Thresholds 

One  of  the  characteristics  of  an  organized  activity  is  that  it  acts  as  u 
transducer  to  convert  negentropy  from  the  environment  into  its  own  organization. 
However,  there  is  a  threshold  of  organization  which  must  be  reached  before  an 
organized  activity  can  assimilate  negentropy  from  the  outside.  This  may  be 
considered  an  analogue  or  perhaps  a  generalization  of  the  noise  reduction  thres¬ 
hold  which  exists  for  organized  signals  in  communication.  In  both  cases,  the 
threshold  represents  a  dividing  region  between  the  loss  and  acquisition  of  a 
standard  of  coherence.  In  the  signal  case  the  threshold  frequently  occurs  in 
the  neighborhood  of  the  value  of  one  bit  of  organized  activity  per  degree  of 
freedom.  We  do  not  have  enough  information  about  other  organized  activities  to 
know  whether  the  same  numerical  value  is  generally  associated  with  the  threshold. 

Not  all  organized  activities  have  thresholds.  Thus,  merely  turning  on  a 
switch  starts  the  activity  of  various  electrical  appliances  used  in  the  home  and 
the  operation  of  the  switch  is  unrelated  to  the  level  of  organization  of  the 
activity.  The  difference  here  is  that  activity  of  these  appliances  is  externally 
driven  and  is  not  self-maintained .  The  examples  in  Sec.  12b  on  the  other  hand, 
which  did  show  thresholds^ were  self-maintained.  There  are  some  appliances  which 
are  hybrids,  being  in  part  externally  driven  and  in  part  self-maintained.  The 
activity  of  such  appliances  can  be  extinguished  and  must  then  be  restarted. 

The  electronic  oscillator  described  in  Sec.  12b  is  self-maintai..ad  and  it 
has  no  apparent  threshold.  This,  however,  is  misleading.  Theoretically  a 
threshold  does  exist  in  this  case,  but  the  threshold  is  so  close  to  the  (2X~kT) 
thermal  energy  at  the  natural  frequency  of  the  circuit  that  thermal  fluctuations 
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The  work  of  Prigogene  [16]  is  pertinent  to  the  present  discussion.  He  shows 
that  stable  macroscopic  mechanical  and  chemical  activities  can  arise  from 
microscopic  fluctuations  provided  that  a  source  of  av. liable  energy  is  present. 
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will  start  the  oscillator  without-  any  significant  time  delay,  when  the  power 
supplies  are  connected  It  thus  appears  that  whenever  the  activity  itself  takes 
part  In  the  transformation  of  external  ne&entropy  into  the  organized  activity, 
there  is  a  threshold. 

The  region  between  the  stable  point  and  the  thresholds  in  Fig.  10  may  be 
called  the  region  of  stability.  This  region  is  determined  by  the  structures 
and  other  constraints  which  determine  the  organization  of  the  activity.  In  the 
design  of  the  system,  this  region  will  be  made  largo  enough  to  handle  normal 
fluctuation  of  the  activity,  plus  a  safety  factor.  Deviation  of  the  actual 
system  from  the  design  specifications  of  the  structures  and  constraints  may  be 
considered  a  kind  of  identity  or  location  noise  in  the  system.  This  noise  will 
cause  a  contraction  in  the  region  of  stability  by  moving  the  thresholds  closer 
to  the  stable  point.  When  the  noise  gets  large,  there  may  be  no  stable  region 
left. 

SPECIAL  METHODS  AND  DEVICES  TO  IMPROVE  ORGANIZATION 

A.  The  Need  for  Limiting  the  Number  of  Coherent  System  Elements 

Every  element  in  an  organized  system  has  a  certain  amount  of  noise  associated 
with  it.  If  the  noise  happens  to  be  white  gaussian  noise,  then  the  noise  power 
level  of  the  amplitude  noise  in  the  system  is  proportional  to  the  number  of  sys¬ 
tem  elements  Whether  the  noise  is  white  gaussian  or  not,  if  the  number  of  system 
elements  is  so  large  chat  the  noise  level  exceeds  some  noise  reduction  threshold 
in  the  system,  then  the  system  is  likely  to  be  useless.  This  is  one  reason  why 
it  is  desirable  or  necessary  to  limit  the  number  of  coherent  elements  in  an  or¬ 
ganized  system 

Location  or  identity  noise  provides  what  is  perhaps  an  even  more  important 


reason  for  limiting  the  number  of  svscera  elements  between  which  coherence  is 
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required.  In  Section  7  it  was  pointed  out  that  in  ordered  sets  of  elements, 
location  noise  sets  an  upper  limit  to  the  range  over  which  it  is  possible  to 
maintain  coherence.  Whether  the  elements  are  ordered  or  not,  we  may  expect  the 
same  type  of  mechanism  to  limit  the  number  of  system  elements  between  which  it 
is  possible  to  maintain  coherence. 

The  foregoing  paragraphs  indicate  two  reasons  why  the  number  of  elements 
which  can  be  effectively  organized  in  the  same  coherent  system  is  limited.  The 
numerical  value  of  the  upper  limit  depends  upon  the  noise  level  per  element  of 
the  amplitude  and  identity  noise.  Because  of  this  limit,  however,  the  design  of 
large  organized  systems  is  usually  arranged  so  as  to  limit  the  number  of  elements 
between  which  it  is  necessary  to  maintain  coherence.  Two  methods  for  doing  this 
will  be  discussed  in  (B)  and  (C) . 

B.  The  Separation  of  Activities  to  Improve  Coherence,  Specialization 

In  a  large  organization  or  a  large  project,  it  is  customary  to  assign  the 
different  tasks  that  must  be  done  to  different  parts  of  the  organization.  This 
Is  an  example  of  what  we  talked  about  in  the  preceding  paragraph.  This  relative 
isolation  of  separate  functions  reduces  the  number  of  system  elements  between 
which  coherence  must  be  maintained  and  tends  to  reduce  confusion  and  interference. 
We  see  examples  of  it  everywhere. 

In  biological  organisms,  such  as  man,  this  isolation  of  different  functions 
takes  place  both  in  space  and  in  time.  The  separation  in  space  is  Indicated  by 
the  fact  that  different  organs  have  different  functions.  The  separation  in  time 
is  illustrated  by  cyclic  behavior.  Very  frequently  this  is  a  separation  into  a 
period  of  activity,  followed  by  a  period  of  rest  and  recovery.  The  physiological 
activity  during  the  two  periods  is  quite  different,  as  is  illustrated,  for  example, 
by  the  very  different  physiology  in  the  depolarization  and  repolarization  of  the 


heart . 
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C.  Bandwidth  Reduction  and  Its  Analogues.  Cells 

If  specialization  were  described  as  horizontal  separation,  then  the  analogues 
of  bandwidth  reduction  might  be  described  as  vertical  separation.  The  separation 
of  an  crganism  into  cells  is  an  analogue  of  bandwidth  reduction.  The  activity 
which  takes  place  inside  a  cell  has  coherence  (organization)  between  a  large 
number  of  degrees  of  freedom.  Large  as  this  number  is,  it  is  still  small  enough 
so  that  the  organization  can  be  kept  above  the  noise  reduction  threshold.  This 
would  not  be  possible,  if  it  were  attempted  to  operate  all  the  degrees  of  free¬ 
dom  in  all  the  cells  of  the  body  in  one  coherent  organization.  Each  cell  inter¬ 
acts  with  the  rest  of  the  body  in  terms  of  a  relatively  small  number  of  degrees 
58 

of  freedom  .  Thus,  for  example,  a  thyroid  cell  accepts  stimuli  for  the  release 

of  thyroid  hormone  and  responds  by  releasing  the  hormone.  It  also  takes  in 

nourishment  and  releases  waste  products.  It  is  likely  that  it  also  interacts 

with  neighboring  cells,  in  a  manner  which  is  not  well  understood,  to  coordinate 

their  external  structures  and  activity.  These  interactions  of  the  cells  with 

the  rest  of  the  body  are  related  in  a  coherent  organization  throughout  the  whole 

body.  Since  the  number  of  external  degrees  of  freedom  per  cell  is  relatively 
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small,  it  is  possible  to  keep  this  organization  coherent  throughout  the  body 
A  similar  vertical  separation  takes  place  in  human  social  organizations, 
and  for  the  same  reasons.  Thus,  a  large  company  is  separated  into  different 
levels  of  organization.  The  different  compartments  of  the  organization  are  in- 
terrrelated  by  a  relatively  small  number  of  degrees  of  freedom,  as  compared  with 
the  interrelations  between  individuals  in  a  given  compartment. 

D.  Redundancy 

w  —  1  .  -  -fa- 

1 .  Redundancy  for  Error  Reduction 

Since  all  organizations  are  subject  to  stochastic  disturbances,  it  is  very 
common  to  use  redundancy  to  Insure  that  important  things  are  done.  Thus,  in  the 

5  8 

It  is  this  reduction  in  number  of  degrees  of  freedom  which  is  the  analogue  of 
^bandwidth  reduction 

The  bandwidth  reduction  actually  takes  place,  at  least  in  part,  in  more  than 
one  step,  i.e.,  room  cell  to  organ  and  then  from  organ  to  entire  organism. 
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body,  there  are  multicellular  tissues.  If  one  cell  fails  to  respond  to  a  stimulus 
applied  to  the  tissue,  this  is  not  important  as  long  as  other  cells  do  respond. 
There  is  even  redundancy  in  entire  organs.  Thus,  there  are  two  eyes,^  two  ears, 
two  lungs  and  two  kidneys.  There  is  also  redundancy  in  type.  Thus  fat  and 
carbohydrate  can  both  be  used  as  energy  sources  and  when  necessary,  glucose  can 
be  obtained  from  protein. 

2,  Redundant  Use  of  the  Same  Materials  and  Methods  -  Standardization 
The  storage,  procurement  and  training  problems  of  an  organization  can  often 
be  simplified  by  using  the  same  materials  or  the  same  methods  for  ?.  variety  of 
purposes.  This  is  so  well  known  in  human  organizations  that  it  is  not  necessary 
to  elaborate  on  it.  This  use  of.  9 tandardization  reduces  the  amount  of  neeen- 
tropy  (organization)  required  to  accomplish  a  given  task. 

It  is  interesting  that  in  biological  organisms,  this  is  also  commonplace. 
Thus,  the  same  amino  acids  and  the  same  nucleotides  are  used  in  the  body  in  a 
variety  of  ways  Also  the  same  chemical  processes  are  used  in  a  variety  of 
applications . 

E.  Monitoring  of  the  Organization  and  The  Environment 

Because  of  the  unpredictability  of  the  tasks  which  an  organization  may  be 

called  upon  to  perform  and  the  unpredictability  of  the  environment,  the  state 

of  the  organization  as  well  as  the  environment  must  constantly  be  monitored.  It 

is  then  possible  to  take  corrective  action  to  deal  with  defects  and  fluctuations 

from  the  normal  or  stable  state,  and  to  prevent  the  system  from  being  perturbed 

beyond  some  disaster  threshold, 

6 1 

F.  The  Use  of  Carriers 

A  widespread  phenomenon  in  organized  systems  is  the  use  of  carriers ,  The 
vocal  chords  generate  a  sound  carrier  which  is  modulated  in  speaking  so  that  it 
carries  a  message.  Radio  and  television  stations  broadcast  their  programs  by 

^There  are,  of  course,  reasons  for  having  two  eyes  and  two  ears  other  than 
redundancy . 
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Ref.  114)  Chap.  V  also  Ref.  [17] 
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modulating  a  high  frequency  carrier.  Trucks  act  as  carriers  to  transport 
merchandise.  Red  corpuscles  act  as  carriers  for  oxygen  and  carbon  dioxide. 

An  obviously  valuable  characteristic  of  a  carrier  is  that  it  greatly 
increases  the  efficiency  and  versatility  of  transportation.  However,  the  sel¬ 
ectivity  characteristics  of  carriers  are  probably  even  more  important,  especially 
from  an  organization  point-of-view.  Thus,  radio  and  television  broadcasting 
would  be  chaos,  if  the  receiver  could  not  choose  a  particular  program  by  tuning 
to  the  carrier.  In  many  cases  there  are  both  carriers  and  subcarriers  and 
after  a  first  selection  by  tuning  to  the  carrier,  there  is  a  second  selection 
by  tuning  to  the  subcarrier.  This  is  common  in  telephonic  communication  and  in 
pulse  modulation  systems,  but  subcarriers  also  exist  in  other  systems.  When 
merchandise  is  sent  through  the  mails,  the  package  with  the  address  acts  as  a 
subcarrier  ,\nd  the  truck,  train  or  plane  which  carries  the  packages  acts  as  the 
main  carrier.  Without  carrier  selectivity,  transportation  would  be  chaos. 

In  a  very  general  sense,  we  can  say  that  carriers  set  up  coherence  between 
distant  locations.  Another  way  of  describing  the  action  of  carriers  from  an 
organization  point-of-view  is  to  say  that  carriers  carry  negentropy  from  one 
location  to  another. 

We  shall  postpone  a  detailed  analysis  of  the  action  of  carriers  until  we 
examine  their  activities  in  particular  cases  of  interest. 
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APPENDIX  II 


BIOLOGICAL  PROGRAMMING  OF  GENES  AS  RELATED 
TO  THE  BIOGENETIC  LAM  AND  EVOLUTION 

The  program  of  development  of  a  biological  organism  is  inherited  from 
generation  to  generation.  A  remarkable  characteristic  of  this  program  is 
the  "biogenetlc  law",  which  states  that,  "every  organism  in  its  development 
mimics  the  evolutionary  development  of  its  ancestry."  This  law  Is  evidence 
for  the  point-of-view  that  evolutionary  development  is  fundamentally  a 
consequence  of  program  additions  at  the  end  of  the  preceding  program  rather 
than  revisions  in  earlier  parts  of  the  program.  This  does  not  rule  out  the 
possibility  of  such  revisions  as  a  secondary  effect. 

In  our  opinion,  the  foregoing  throws  doubt  on  the  theory  that  evolution 
is  a  consequence  of  stochastic  mutations  and  natural  selection.  Natural 
selection  no  doubt  causes  the  extinction  of  many  species.  However,  the  above 
evidence  indicates  that  evolution  probably  takes  prace  as  a  consequence  of 
additions  to  the  DNA  to  extend  the  program  development.  These  additions 
need  not  take  place  physically  at  the  ends  of  previously  existing  chromosomes. 
It  depends  upon  the  relation  of  the  program  sequencing  to  the  physical  loca¬ 
tions  of  the  activated  genes. 

The  author  has  reported  elsewhere^  investigc  iwing  a  consistent 

similarity  between  the  phenomena  of  biology  and  physics  This  suggests  that 
just  as  the  development  of  the  motion  of  a  mechanical  system  is  predetermined, 
cr.  (ensemble)  average,  by  the  laws  of  quantum  mechanics,  and  the  develop¬ 
ment  of  any  physical  system  is  predetermined  on  the  average  by  the  laws  of 
physics,  so  also  biological  evolution  is  predetermined  on  the  average  by  laws 

[1]  S.  Goldman  "The  Mechanics  of  Individuality  in  Nature,"  Foundations  of 
Physics  1,  p.  395  (1971).  A  continuation  article  with  the  same  title 
has  been  accepted  by  the  same  journal. 
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of  biology.  Just  as  in  physics,  one  would  expect  a  statistical  distribution 
in  the  probabilities  of  the  alternative  paths  of  evolution,  but  these  alter¬ 
native  paths  and  their  probabilities  are  determined  by  laws  of  biology,  even 
though  these  laws  are  at  present  unknown  to  us.  If  this  is  true,  then  evo¬ 
lution  is  not  a  consequence  of  chance  mutations  and  natural  selection  in 
the  complete  sense  that  has  been  generally  accepted.  It  is  accordingly 
doubtful  that  chance  mutations  induced  chemically,  thermally  or  radiologically 
are  effective  in  evolutionary  development,  unless  they  happen  to  be  part  of 
one  of  the  predetermined  evolutionary  alternatives  mentioned  above. 

The  plant  and  animal  worlds  consist  of  distinct  groups,  for  example, 

(phylum,  class,  order,  family,  genus,  and  species)  and  a  continuous  range 

[21 

of  individuals  with  intermediate  characteristics  has  not  been  found. 

This  is  entirely  in  accord  with  th°  discussion  of  biological  eigenstates  in 
Ref.  [1]  and  the  discussion  of  stability  and  thresholds  in  Sec.  12  c  &  d  of 
Appendix  I.  The  probability  that  a  random  mutation  would  take  place  which 
jumps  the  gap  between  phyla  seems  most  unlikely  even  in  geological  epochs 
involving  billions  of  individuals.  It  seems  much  more  reasonable  that  such 
a  mutation  which  simultaneously  involves  a  definite  pattern  of  an  enormous 
number  of  genes  would  be  largely  predetermined  by  the  as  yet  undiscovered 
laws  of  the  biological  analogue  of  quantum  mechanics. 


[2]  D'Arcy  Thompson,  "Growth  &  Form,"  p,  1093-1095. 


